what is the range of frequencies in usb power supply noise? - it can be from 20Hz to as high as 300Khz
circuit to remove them? - an RC filter at the input of the device could probably solve it, using R = 10ohm C = 0.1uf//10uf. but it's not a sure fire solution.
what device are you trying to power thru a usb and how much current will your device draw?
