Why are you expecting decent voice recordings from the Fuze? it was not designed primarily to be a recorder. They just threw in a cheap mic and some cheap circuitry to support the voice recording. It does work much better when the player is within 2 or 3 feet of the speaker's mouth. Some people clip a Clip or Clip+ to the speaker's lapel or shirt pocket, and then the recordings aren't that bad.
for voice recording, I suggest that you get something that is primarily a recorder. Olympus makes some inexpensive ones that are usable enough for casual use which record in wma.