All audio reviews are fatally flawed. Hello - did I just wake you up? Sorry, categorical statements tend to be a bit shocking, I admit. This one happens to be 100% factual for a change. Here's why. Consider a basic system of
CDP interconnect preamp interconnect amp speaker cable speakers.
We're looking at 7 hardware variables plus room interaction. Say a new preamp arrives. You insert it into said chain and report on the differences. Since only one variable changed, the final audible differences are clearly due to the newcomer. End of story, check's in the mail. Right?
Wrong on both counts. For illustration's and argument's sake, let's assign the value of 'Zero' to neutrality. Increasing positive values shall denote shifts into undue warmth/romance, increasing negative values a skewing into unnaturally heightened resolution/detail. Let's further state the obvious, namely that the ideal reviewer system would arrive at neutrality through a combination of all neutral components. Our chain reaction would work out as follows:
0 0 0 0 0 0 0 = 0
If our new preamp deviated even the slightest bit from neutral, the sum total on the right would reflect the precise amount of deflection. So far so good - if systems like that existed. Unfortunately, they don't. Instead, let's consider a real-world rig which is still idealized in our assumption that the value to the right of the equal sign would amount to precisely zero - or that anyone alive outside Harry Pearson would recognize the precise moment when neutrality was approached, then achieved, then hit the breaks to lock things into place for good:
1 -2 -1 0 3 1 -2 = 0
This example shows how over time and with plenty of trial and error, the reviewer has assembled complimentary components to offset specific deviations from the theoretical median and, despite their individual flaws, used subtraction and addition to tally up as neutral in the end. (Of course, this is still assuming he'd recognize neutral if it bit him in the arse.) For argument's sake however, let's give him that. Bloody well done, old chap. Jolly good. You're a Zero. Cellophane man!
What we cannot give our self-congratulatory chappy under any circumstances is the key to our car, nor the irrefutable certainty about what component in his system is doing what, exactly. None of them produce sound on their own and thus can't ever be tested in isolation. This is so obvious as to become obscure again. So while there is a faint chance that one could arrive at 'zero' or neutrality, there's not even a slim chance in hell of ever determining the exact values each component in the chain imposes on the one preceding it, the one following it and all of them in concert, together. To make matters worse, remember now that my arbitrary value system operates between merely two poles. That's clearly not sufficient to account for the variety of contributing qualities that make up an audio system.
Alas, before our discussion becomes stillborn due to obfuscation and unnecessary complexity, let's take our second 'equation' at face value. This is the system we're using as our de facto reference system:
1 -2 -1 0 3 1 -2 = 0
Let's further assume that our new preamp arrived with a neutrality score of 0 - except we don't this yet. As a reviewer, we're supposed to do due diligence and find out. To ascertain its neutrality factor, we insert the new component into the system like good little reviewers do. We dutifully notice that the final tally deflects to +1 (our resident preamp was a -1 but we can't know this). Using our previous logic and merely accounting for the final offset, we pronounce this preamp to be +1, hence slightly on the warm side of things. To be honest, we can count ourselves very lucky that the preamp it replaced merely suffered the -1 value, hence our final verdict deviated from the truth just a little, from 0 1. Whew, close call - but then, nobody (including ourselves) knows better so we're cool!
Can you appreciate the rather gross error we'd have committed had our power amp traded places with our resident preamp? Replacing a +3 preamp with a 0 would have skewed the final figure from 0 -3. We'd have declared the same piece of hardware cold, analytical and severely lacking in warmth and musicality. Not even close, never mind a call. Pure horseshit! Except, does anyone truly know the real value of this preamp to challenge us?
Does this overly simplistic example drive home how it is really impossible to perform objective assessments? There's simply no way that anyone could ever definitively know where and how any of his components deviated from neutrality. While the desirable outcome of neutrality is arguably within reach, the road toward it remains obscure. Regardless of how you begin to build a system, you're always the victim of reacting to the interactions of your first components. That's how you determine what to get next. If a new acquisition scored high on improving 4 vital parameters but diminished one, you'd get a tweak or accessory to address the one loss. In all likelihood, said tweak would do more than just surgically remove the previous component's isolated negative - and the endless juggling for balance continues ad infinitum thereafter.
The fact remains, you only recognize deviations when you come across something that deflects the needle to a lesser degree. Such encounters are entirely arbitrary. Even if you were completely committed to this search, you can never know a priori what a new component might do. Desire and ability do not walk hand in hand here. How to know which component to go after to solve your dilemma? The solution is to become a whore - sleep with everyone. If you try enough components for long enough, you'll go broke -- at least the whore makes a living -- but you might eventually come across one piece of gear that fits your situation like that fabulous missing piece of the puzzle, precisely and without any overlap or empty spaces. Until, that is, a chance encounter demonstrates further limitations you weren't aware of until they were removed. Or introduced you to a quality that wasn't even on your radar screen yet. Or introduced you to the folly of believing that your system added and subtracted to a final 'zero' or 'neutrality' to begin with...
Put plainly, in audio writing performed with sincerity, there's simply no substitute for experience, exposure and full accord with the innate dilemma of the profession, plus humor and honesty about it. At best, audio reviewing becomes an educated guess; a hunch of likelihoods; a bunch of descriptions about how insertion of a component deflected that system, in that room and to those ears, and what that might indicate about the component itself. If these matters were indeed so -- and make no mistake, they are -- what service can audio reviews actually render outside of identifying operational flaws and act as entertainment or printed toilet paper? Ah, time for the commercials...