YES! Cover the brushed...
Keep polishing your areas a little longer, it should go to a nice mirrored polish.
I like to use different parts of the pink cloth as it does start to 'tarnish' the cloth - I don't really know what that is or why, but I just move on to a different part of the cloth.
I've done it to numerous different polished surfaces, with a little time, each has been restored to its former glory.
Of course, yes, you care removing a layer, so I wouldn't do it to the same piece often.
Chris - Numismatist