Admittedly, my personal USB microphone experience is limited to a USB headset mic I have for Skype. I have tested it out for recording purposes and was not impressed with the quality.
I’ve heard some recordings made with desktop USB microphones, though, and found the listening experience to be pretty decent. One example that comes to mind is some recordings done with a Snowball mic by BlueMic.
The problem with USB mics is that they require a computer; they take away your portability unless you plan to carry your computer around with you (much bulkier than a portable digital recorder). And, in the case of the headset mics, I find that people who use them for recordings tend to place the microphone directly in the path of their breathing. If you do use a headset mic, I recommend placing the microphone slightly above the mouth and forward enough that it’s out of the way of both your nose and mouth breathing.
Particularly with USB microphones — which are generally consumer grade for affordability — be sure you test drive a few before you make your final purchase decision. Spending a few extra dollars for a better recording the first time is better than jumping in with cheap gear to start and then eventually moving to something that can be listened to. The audio quality forgiveness period in podcasting has long passed.
