Fear & Loathing in the Audiophile Community: Parte Terza

Based on the responses to my recent articles on what ails the audiophile community, it would appear that my thoughts hit too close to home for some. The comments were a mixed bag of agreement and vitriol; and were a highly illustrative microcosm of exactly what I was referencing.

The most colorful vitriol came from those who have decided that I am an abject (the horror, the horror) subjectivist who doesn’t believe that measurements matter. A quick search of my reviews would have made those same people realize that I am not going to fit cleanly into either of those two camps.

My hope is that clarifying my position will help others understand that my goal is not to make this personal and not to pick a winning side. If you love music and better sound quality — you are already on the winning side.

I’d love to see a community where we can disagree civilly and focus on the things that help us move forward in this hobby.

This is a hobby, remember?

Censorship is never going to work but civility needs to be the guiding principle.

If you think bashing a product or another audiophile who likes that particular product makes you some kind of savior of our sacred realm — check your stupidity and arrogance at the listening room door.

I believe measurements can tell us a lot and I do think they are both useful and needed.

However; I do not think they are capable of telling us everything we need to know. And some things are so dependent on external factors that measurements mean little as temperature, humidity, the shape of the human ear, the size of the room, objects immediately around the listener, and many other items too numerous to mention here will never be exactly as they were during the measurement test.

They are at best in those cases, approximations of what one might expect and at worst entirely misleading.

I am also not willing to trust my ears entirely as auditory memory is exceedingly short and A/B testing is extremely difficult to do well without a team helping (it is almost impossible to blind test speakers by yourself).

I do think an awful lot of what people hear can be chalked up to what they want to hear for a variety of reasons when doing such testing. While some will point to switch boxes or other devices that make it easier to A/B test, those too can add additional noise and distortion to the mix. 

There are a number of reasons why I can’t sit completely in either camp.

I’ll start with measurements but promise to give equal time to subjectivism so stay with me even if you don’t like what I have to say — I’m speaking directly to the Audio Science Review crowd.

Even the simplest of measurements done by hobbyists are dependent on how well calibrated their gear is and how accurate and precise those instruments are. It has long been warned that frequency response charts made on one system cannot be compared to those made by different systems.

Even with proper calibration, graphs I have done will parallel the curves of graphs published by Head-Fi or ASR but not directly overlay them. The shape is the same but minor differences almost always exist.

I’ll be the first to admit that both Jude and Amir are likely better at it, have more money invested than I do in proper testing equipment, and have more practice doing it.

The fact that their two graphs don’t agree entirely though leads me to the conclusion that even among experts with high end measurement systems there is no real consensus, and one could rightly question why there are these differences in their reporting.

Other questions about measurements center on the statistical analysis of those measurements. How many times is a test run to calculate a result?  If the same test is run 10 times consecutively do we get exactly the same result or is there variance?  

What is the margin of error? If a SINAD measurement has a  ±2% margin of error, then items between 110 and 114 dB may all actually perform exactly the same since the item rated at 110 dB may have a 2% error to the negative and the item with the 114 dB rating may be an equal amount off in the opposite direction (roughly 110 dB + 2 dB; 114 dB – 2 dB). 

Again, these numbers are bantered around like gospel but without context. They mean little and can be misleading as a result.

When was the last time you saw a measurement session documented that listed how many times the tests were performed, the raw data and the averages, and the calculated margin of error for those tests disclosed?  

Sometimes measurements are misrepresented accidentally because of differences in language, translations, or just plain differences in our understanding of what each measurement means.

Take power output as one example. Some will say that looking at the milliwatts of output of an amplifier at a given impedance is basically all you need to know in order to calculate how well a headphone can be driven by that amplifier.

Others argue that since wattage is a calculation based on current and voltage, you really need to know the component parts that make up that value to be able to determine how well the amplifier is suited to work with a particular headphone. 

Averages are often stated and yes, the Harman curve is a big one. By using averages we increase the likelihood that a product is suited to a large group people but very few of us are fit perfectly in that average.

As an example, I used 10 years of test grades from a local institution and compared students actual grades to the average; only 3% of the time in that ten year period did a student score exactly the average grade on any given examination. In many cases, the act of averaging the data made it impossible to score exactly the average.

An average may provide a rough estimate of individual performance, but it rarely is a match to actual performance.

And then there is the fact that science as we know it changes daily as new discoveries are made about the human brain, hearing, how sound waves are propagated through various media, and the technology available to us evolves.

Over the millennia, we have been sure that the Earth was the center of the universe, spontaneous generation occurred (rotting food gives rise to maggots), the universe was stationary, and mass was lost during chemical reactions.

These were not ideas proposed by ignorant men, with most being proposed by some of the most renowned minds of their day. The Stationary Universe Theory was put forth by none other than Einstein himself, only to be proven wrong by Hubble.

My point is that science is the best that we understand the universe around us today, but that is subject to change as we move forward. Saying “this cannot be true because science says…” is denying that science evolves.

Sure it may not seem true based on what we know, but even the law of gravity is subject to exceptions that were not known in Newton’s day.

And now for the flip side of the argument. 

I cannot be a subjectivist solely, because I see too many flaws and fallacies being used to justify statements.

You Can’t Prove Me Wrong, So I Am Right…

The burden of proof always lies with the person making the claim and too often the follow-up to this first bit of nonsense is, “just because you can’t hear it doesn’t mean it isn’t there” to which I would say, ”just because you think you hear it, doesn’t mean it is either.” 

The first argument; “you can’t prove me wrong, so I am right” is Argumentum ex Silentio or to argue from silence. As the phrase suggests, it has been going on since the days of ancient Rome, and says if current sources cannot speak to a question it proves a person’s claim.  

The problem being is that it works equally well in both directions. “Science can’t prove God exists, so he doesn’t,” is equally valid when compared to “Science cannot prove God doesn’t exist, so he does.”

My example may be seen as extreme, but it shows the fundamental problem with arguments from this position.  

Secondly, the “just because you can’t hear it” line of thinking automatically assumes the claimant is in a better position to judge than the other person in the conversation without presenting any evidence to support the claim and it relies on a sample size of two.  

To prove a hypothesis, a result must be repeatable and that pretty much precludes using only the sample size of “you.”

EIC Ian White would argue that the only opinion that matters is your own based on your own listening experience and that audiophiles should just buy more music and enjoy the hobby. You either like how the music sounds in your own specific listening space or you don’t. Nothing else should matter.

When presumed differences in a listener’s capabilities are called into the argument it seems scrutiny is always placed on those who don’t believe but never on those who do. If 99 out of 100 people don’t hear a difference and you do, chances are your hearing is not better; you have a problem — time to get it checked.

Another problem I have is the confusion between different and better. Changing a component may indeed result in differences in the sound. Swap your speaker wires for a single 28 gauge strand of copper (easy and cheap to get, just use 1 strand of an Ethernet cable) and you will almost invariably change the sound because resistance and impedance is raised considerably.

But what makes that change “better?”

There is an age-old fallacy known as confirmation bias that comes into play here. If we paid more for it, and we hear a difference, it must inherently be an improvement because we expect spending more money to equate to a better outcome.   

We are conditioned to think “you get what you pay for” but in audio it is entirely possible to get a whole lot less.    

Blind belief that any change is improvement is a sure way to part with a lot of your hard earned cash. As an example, some manufacturers have started intentionally introducing distortion (second order harmonics) to tones below 100Hz in their smaller Bluetooth speakers because it adds what we perceive as greater bass impact.  

This is done because small drivers struggle to produce deep bass. The most common way to improve low end response is to increase size of the driver, size of the vent, and size or the cabinet.  

None of these options are really that practical when designing a portable speaker so designers have to resort to other means. Does this really result in something that is better?

When we don’t have common definitions of what is better, or improved, it is near impossible to state unequivocally that any change results in an improvement, but instead only that it results in change.

Cult of Personality

There are two other camps in the audiophile community that push people away. If you believe that a single person is the ultimate source of the truth; whether it is a successful designer, reviewer, or forum moderator — and you are willing to die on that hill defending their positions, you are no longer thinking for yourself.

Trust your own ears. Why would anyone blindly follow the conclusions of anyone who has never engineered a single respected or successful piece of audio or video equipment.

Even the best designers release the occasional dud, but to think that someone with a fancy lab of test & measurement equipment knows more than them is a weird line of logic to follow.

I would also recommend that people new to the hobby ignore the bandwagon jumpers on most A/V websites.

When you surround yourself with only people who believe the same things you do, chances are, you are going to have those views reinforced, and believe they hold more validity than they do. If you only frequent one forum, or one site, and it has a reputation for being objectivist or subjectivist, chances are what you are reading already has a bias and you are inheriting it.  

We have fanaticism on both sides of the aisle and that isn’t good science as fanatics tend to overvalue things that support what they already believe and discount the things that differ from their viewpoint.   

Failing to give all the evidence equal weight makes us look no different than the sports fan who screams at you online that their team is going to win the Stanley Cup when they currently sit in last place.

Online trolls love to ask hot button questions that they know will create a fight and our best defense is to just ignore them. Our hobby has a lot to offer people but we have to question our behavior when it continues to polarize or scare people away.

Happy New Year.

Continue reading:

← Part 1: The Audiophile Community has a Problem

← Part 2: The Audiophile Community has a Problem