Sansa Clip+ vs iPod Touch Detailed Measurements & Review

[esanthosh]

Found this in head-fi just now. The blog post pits Clip+ against iPod Touch 3G - http://nwavguy.blogspot.com/2011/02/...-measured.html

Though I can't understand much of it, I think Clip+ does better by a big margin.

[CrowdAds]

[Marvin the Martian]

I'd love to see the Hifiman tested against the Clip+.

[dfkt]

Yeah, with that gear, that would be even more devastating than my measurements.

[NwAvGuy]

I've heard good things about this forum and like what I've read so far. It's kind of amazing how much market share Apple has. And it's interesting to compare the alternatives.

For anyone interested, I've conducted detailed measurements of both the Clip+ and iPod Touch 3G using a high-end professional audio analyzer that goes well beyond the usual RMAA measurements (which I also use). Does the Clip+ live up to its reputation when it's really under the audio microscope?

The full review is on my personal (non-commercial) blog and includes a summary followed by lots of detailed measurements and some background into the "how and why" of measuring a portable digital audio player. I also try to discuss how several measurements can affect the audible sound quality.

My blog also has some other articles with tips on headphone amps/DACs, how to get the best results from RMAA, etc. Look for more upcoming reviews on players (the Cowon iAudio 9 is next), headphone amps (a FiiO review is coming soon), headphone DACs (the NuForce uDAC 2 is sitting here begging to be connected to the analyzer), and several in-ear headphones as well.

My blog: http://nwavguy.com

[esanthosh]

I already posted the link yesterday (as soon as it was on head-fi) - http://www.anythingbutipod.com/forum...d.php?p=539071

[NwAvGuy]

@esanthosh thanks for posting the link.

I'd be happy (and more than a little curious) to test a HiFiMan with the dScope, Benchmark ADC1, etc. But their players are too big and heavy for my tastes in a portable and too expensive to buy just for testing purposes.

If the HiFiMan specs are honest, they have a clear advantage in output level. But that only matters if the Clip's generous output isn't already enough.

Otherwise, I suspect any audible differences are due to the HiFiMan's (alleged) rolled off response with the Clip+ actually being more accurate and true to the source material. The HiFiMan may also have a higher output impedance. I've not seen a spec or measurement for that. And a higher impedance would also degrade the accuracy of the sound (especially with certain headphones) and might make for some additional audible differences.

[dfkt]

Yes, basically the only (audible) thing the Hifiman has over any other quality players is its output level, not much else. The treble roll-off of the Hifiman is very real: http://rmaa.elektrokrishna.com/Compa...0vs%20H300.htm - it's also audible in volume-matched AB and ABX tests.

While I don't have your means to measure output impedance properly, an RMAA test with some picky multi-armature phones is already hinting at its very high output impedance: http://rmaa.elektrokrishna.com/Compa...Impedances.htm Same tests with the Clip+: http://rmaa.elektrokrishna.com/Compa...Impedances.htm (Multi-armature IEM loads tested at low volume, to not damage them - only the dynamic driver V-Moda Vibe was measured at full volume. Sound card for all tests was an Echo AudioFire 4.)

Here's some more, although in Japanese: http://sonove.angry.jp/HiFiMAN_HM801.html

[musichound]

Originally Posted by dfkt

Yes, basically the only thing the Hifiman has over any other quality players is its output level, not much else.

So does this mean the Hifiman will deafen a person while listening to its mediocre sound?

[lestatar]

Wow. I am duly impressed. Great great stuff, though most of your tech breakdown went right over my head, I got the gist of what you discovered.

So, indeed, Sandisk really did hit a home run

I am interested to hear what the much more knowledgeable folks here think of your work...

cheers and kudos again,
-les

[NwAvGuy]

@dfkt thanks for the links. Yeah, the HM-801 looks to have a high output impedance and that's sad (in my opinion). If true, I think there's little excuse for it in such a high-end audiophile player unless they did it on purpose to intentionally make it *less* accurate. But I think most would prefer the frequency response of their player not be altered by as much as +/- 5 dB just by using different popular headphones?

But, sadly, may audiophile businesses have learned "different sells". So many are making products intentionally less accurate, and with more audible distortion, just so people can readily notice they sound "different" from more mainstream products. Never mind that "different" is often "worse"--the psychological aspects cover that up nicely.

It's not unlike expensive wines. People prefer the obviously different expensive ones when they know what they're drinking. But put them all in plain brown paper bags, and the more normal cheap mainstream wines often win contests--even with expert wine critics doing the tasting.

Personally, I'd rather have a player that's as accurate as possible, and if I want different sound, use EQ (especially the impressive EQ offered by Rockbox on the Sansa's, etc.) to tweak things to my liking instead of being stuck with say random interaction between my headphones and the player as the HiFiman 801 apparently offers.

[lestatar]

^^ Only when used in conjunction with SkullCandy 'phones...

[esanthosh]

@NwAvGuy,

I know it's difficult to get a Hifiman for the tests, but can you try and get something like S:Flo2 ($150-180) or even the Cube C30 ($50+) for your tests? Both are reasonably priced compared to Colorfly C4 or HM801 / 602. But the repeated claims of "better SQ" make me wonder if it's just the higher output level as in the case of 801.

[NwAvGuy]

Originally Posted by esanthosh

@NwAvGuy,

I know it's difficult to get a Hifiman for the tests, but can you try and get something like S:Flo2 ($150-180) or even the Cube C30 ($50+) for your tests? Both are reasonably priced compared to Colorfly C4 or HM801 / 602. But the repeated claims of "better SQ" make me wonder if it's just the higher output level as in the case of 801.

The Cube C30 could be interesting. It's great when an inexpensive "under dog" product can run (or out run) the "big boys". But, that said, my experience with the no-name Chinese designed products is they usually put more effort into making them *look* and *seem* impressive rather than doing a great job with the stuff that really matters that people often can't see.

Even supposedly "high-end" audiophile Chinese designed no-name products that have impressive looking circuit boards and use trendy components are often half-baked in their implementation. I know one that looks great in the pics on eBay, but they used a poor PCB layout with little attention to proper grounding. It has audible noise issues and measures very poorly.

If you go attend CES and sneak off to the Asia B2B part of the show, there are hundreds of booths of Asian vendors selling things like the Cube C30 hoping to get someone like Logitech to brand and sell their products. But, generally, the products all have one or more fatal flaws that are hard to overlook.

Generally, if something is *only* sold on eBay it's a sign they couldn't find anyone else to sell it. And, in this business, that's generally a bad sign.

All that said, there are exceptions to the rule. And it might be fun to find out if the C30 is one of them? The C4, however, is expensive enough I'd be more reluctant.

[meetooman]

Thanks to the poster and the writer. Impressive effort not to hard to read.

@ NwAvGuy:
Would you consider useful/meaningful to compare encoded files on the same device to create a quality index value? Some people will never agree about Flac vs MP3 on the Clip+, but I would like to know if good enough can be spotted with your testbed.

I don't want you to start a civil war between formats partisans, I wonder if there is a threshold or a sweet spot in quality settings (between say: the Vorbis Q0-Q10 range) that your objective results could show.

I imagine a computed value from weighted results compared to the original wave measurements. Example:
Wave: 100
Codec-A/Q3: 78
Using codec A with preset "3" you got 78% of the wave quality.

[Enigmatic]

Originally Posted by meetooman

Would you consider useful/meaningful to compare encoded files on the same device to create a quality index value? Some people will never agree about Flac vs MP3 on the Clip+, but I would like to know if good enough can be spotted with your testbed.

I don't want you to start a civil war between formats partisans, I wonder if there is a threshold or a sweet spot in quality settings (between say: the Vorbis Q0-Q10 range) that your objective results could show.

I imagine a computed value from weighted results compared to the original wave measurements. Example:
Wave: 100
Codec-A/Q3: 78
Using codec A with preset "3" you got 78% of the wave quality.

The way to test transparency is to ABX.

http://en.wikipedia.org/wiki/Transpa...compression%29.

[NwAvGuy]

Originally Posted by meetooman

Thanks to the poster and the writer. Impressive effort not to hard to read.

@ NwAvGuy:
Would you consider useful/meaningful to compare encoded files on the same device to create a quality index value? Some people will never agree about Flac vs MP3 on the Clip+, but I would like to know if good enough can be spotted with your testbed.

I don't want you to start a civil war between formats partisans, I wonder if there is a threshold or a sweet spot in quality settings (between say: the Vorbis Q0-Q10 range) that your objective results could show.

I imagine a computed value from weighted results compared to the original wave measurements. Example:
Wave: 100
Codec-A/Q3: 78
Using codec A with preset "3" you got 78% of the wave quality.

That's a *really* good question, and something I may try to cover in a future blog post. There are two very different answers to different parts of your question: MP3 vs FLAC

Unfortunately, when it comes to *lossy* compression like MP3, it's all about subjective perceptions. Even my test gear (or anyone else's) can't provide much meaningful objective measurement of CODECs. I laugh when I see RMAA data showing MP3 vs WAVE (or FLAC) data. It's meaningless.

The whole idea behind lossy CODECs is to throw out information that's not likely to be noticed by a human as missing. But that's a very subjective thing that pretty much requires a human to interpret. Test equipment has no effective way to distinguish between what's likely to be missed and what isn't. And even the CODECs themselves are designed with blind listening panels not test equipment.

For the 2nd part, FLAC, I can tell you there is honestly *zero* difference between FLAC (and other lossless compression) and a CD quality WAVE file. None. If someone hears a difference, it's either imagined or due to some other variable besides the file type.

I can say that with such certainty as it's very easy to prove the bitstream entering the DAC is identical in either case (or at least should be as it's trivial to do so). In either case you're pulling a sequence of numbers out of a file in an asynchronous (non time critical) way. The only difference is you run the FLAC numbers through some added processing that turns them into the original WAVE numbers before sending them on to the DAC. The DAC honestly can't tell the difference. And if it's getting an identical bitstream, the sound has to be the same either way.

Some will try to tell you the jitter is different, etc. But that's all myth, not fact. To put it another way, anyone can do this on their PC:

• Make a FLAC file from a WAVE file (now you have 2 files)
• Convert the FLAC file back to a WAVE file (now you have 3 files).
• Compare the converted WAVE file to the original WAVE file using a file comparison utility and verify they're identical (i.e. "bit accurate"). Foobar 2K can do this.

The answer will always be they're identical. The same thing happens playing back a FLAC file it's just the conversion takes place in the player.

Finally, there *is* one interesting method (besides the usual listening tests--blind or otherwise) to compare lossy CODECs. It's called "audio differencing" and it subtracts the compressed signal from the original to let you hear what was removed. There are various ways to do this, but a popular one is:

http://www.libinst.com/Audio%20DiffMaker.htm

The above page even has links to ready made difference files and one of them compares MP3 to the original uncompressed format. By listening to the difference you can at least have a somewhat better idea of what's being removed by different CODECs and/or using the same CODEC at different settings.

[glove4]

I wish the Clip+ would top the Touch 3G in battery life as well.

[esanthosh]

^^ I am not worried about the 'last 2-5%' that people seem to spend loads of money on. But, many of the so called high-end (as in costly) products highlight better jitter correction as one of the features. How much does jitter affect the final output - as in comprehensible to an average listener like me?

[meetooman]

Thank you for the answer. But it is obvious for me that comparing Flac to wave is not worth your time.
My concern is whether compromises made when using lossy encoders at different bitrates can be measurable in a useful way using key parameters (channel separation, frequency response, distortion).
I know many people only trust listening, but I would like to discover is which way (linear or not) different bitrates are giving better or lower results. Then, if it would be possible to draw eventually a quality/file size graph showing us the "sweet spot". Will that spot change with each different music?

AudioDiffMaker seems to be a very interesting tool for ABX test fans. Do you think we could compute an "deviation" value out of a "difference" system?

[skip252]

Originally Posted by meetooman

AudioDiffMaker seems to be a very interesting tool for ABX test fans. Do you think we could compute an "deviation" value out of a "difference" system?

Instead of wondering you could do a little research and let us know what you find. You could start with some of the videos on Youtube and an interesting thread over at Hydrogenaudio on comparing the difference between the original lossless file and a lossy encode.