|Say you're a seasoned audiophile. Over the years, your system has become more and more optimized until each component works ideally with all the others. From a reviewing perspective, you've naturally painted yourself in a corner. Taking out one component for another will, nearly be definition, have to upset the very balance you have so carefully cultivated. Your opinions on the new arrival will be premature - until you've reestablished that precarious balance all over again.
As those cynical from experience know for putting in a new component, 10 steps forward, none back is pure theory and wishful thinking most of the time. Even with plenty of illicit trysts before committing -- pre-auditions, something many audiophiles can't do -- one is well ahead in the real world if one scores eight steps forward and only two back. And yes, infatuation needs to make room for a cooler head before any backwards steps will be noted and acknowledged. What is true for audiophiles remains doubly true for reviewers. Which raises certain questions. How can a reviewer create a best-case scenario for any review component unless the resident tool chest was rather stocked with alternate mates and enough patience at work to try out the gamut of potential combinations on hand?
It's obvious that the more of everything a writer owns or has on long-term loan, the higher the chances that for any given assignment, he'll come close to optimizing the system for the component under review. In some cases, he might even score a perfect 10. But how to amass such an arsenal on the pay grades of low to no in this field? After all, even a true golden ear can only ever listen to one stereo system at a time. There is no innate motivation or reward for owning more than one of anything for personal pleasure. No, the stocked tool chest is simply an eventual requirement of the trade. It's nothing regular hobbyists should try to emulate or be jealous over unless they simply love to mix'n'match as the mood strikes. In which case, they'll take a truly cocreative attitude to the playback experience to reject any fixed standards.
While lots of toys may seem like an attractive perk to would-be reviewers and readers, the truth of the matter is quite the contrary. Once you've tweaked your system the way you like it, any changes are upsetting and rarely if ever more satisfying than what you already have. If pleasure is on your mind rather than anal-yzing, you want to leave your rig as is. Therein lies a key reviewing liability. Unless you have completely discrete systems for pleasure and work (one don't touch, the other in permanent flux), you'll secretly resent the newly arrived review subject. Which is plainly the wrong attitude.
With an open-enough mind and ear willing to experiment with different flavors of sound to get unstuck from the my-way-or-the-highway preference, flux needn't be bad. It can safeguard from getting bored. It does however require enough hardware tuning options to maintain a gourmet standard of different sonic cuisines. Otherwise you might devolve into campus cafeteria land with the old "just change one thing at a time" routine.
That routine of course is hoary credo central of subjective audio reviewing. Just like in soap operas, the moment you change multiple things, you've completely lost track of who did what to whom and why. Which is precisely how synergy works. It transcends linear math and thereby eludes linear tracking. To repeat synergy could require duplicating the original circumstance down to the T. Now we're approaching complete melt-down: reviewing and recommending complete systems where the only unavoidable -- but potentially huge -- variables left are the room and the listener.
If all of this is so (and most career 'philes would concur it is), how useful does this render the usual reviews done up by the usual writers with the usually limited range of match-up hardware? It's a fair question to ask. Everyone in this hobby needs to answer it personally. If you come down to feeling that in the end when everything has been said and read and digested, reviews are primarily for entertainment and to stay in the loop of goings-on, I couldn't disagree with you. There's a Ralph Fiennes flick which reenacts a famous brouhaha of an American pop quiz show that turned out to be rigged by giving the winners the answers before each broadcast. Since the show's income was ad revenues, viewer ratings were important. Attractive non-geeky contestants with supernatural scholarly learning and a fiendish grasp on pop facts drove up those ratings so such contestants were carefully - groomed. At the end of the movie when the show's fall guy producer stands before a grand jury, we overhear the following closing justification: "Your honor, we aren't hardened criminals. We're in the entertainment business for heaven's sake. We're simply entertainers."
It seems to me much of that is true for audio reviewing as well. You do your best to get your facts straights, to be fair and comprehensive and diligent. But any pretension at the truth, the whole truth and so help you Harry Pearson is simply that - a royal pretension. Experienced audiophiles of course know this full well. They take from reviews what can be taken from them (information, impressions and assumptions) while rejecting absolutes and invariables implied or expressed. The problematic readers are those who approach reviews looking for absolutes. There aren't any to be had. Like that Pop Quiz show, the game of audio reviewing is rigged. It's the nature of the beast. It simply can't be helped no matter what we do. The best solution is to make transparent the rigging on which any reviewer dances like a marionette - listener bias, experience or lack thereof, listening room and system context etc. Then it becomes responsible entertainment. Whether our efforts can ever lay claim to being any more than that you must decide for yourself. I already know what I believe about that...