Comments on the state of audio reviews

[aerius]

http://www.diyaudio.com/forums/showthread.php?postid=1310763#post1310763

Of particular interest:

I'd like to return to the larger point in the previous post comparing the level of engineering in the automotive world to audio - looking also at the level of knowledge of consumers, magazine reviewers, and the target markets the manufacturers are aiming for.

In the auto world, $100,000 gets you a very fast car with superb engineering, or a very extreme off-road vehicle. The claim to do both is ridiculous, and enthusiasts know this - but if you know cars, you can get an awful lot of vehicle if you're willing to pay.

Enthusiasts and magazines are aware of the subjective feel of solid-axle, trailing-link/swing-arm, and double-wishbone suspensions, of the torque curves and subjective acceleration qualities of small displacement/large displacement motors, and the feel of turbos and variable valve timing. A few minutes at the wheel, and you can feel these things immediately - scaring the salesman maybe (never as much fun being the passenger) - but the technical characteristics are immediately evident to an enthusiast.

Even the glossy magazines - with all the questions about corruption - continue to describe the technical highlights and road feel of the vehicle under review. They might minimize reliability issues - that's the domain of Consumer Reports with their user database - but they'll still bitterly complain about the BMW iDriveYouNuts interface or the uncertain future of the Jaguar/Ford cars.

Compare this to the abysmal writing in the Big Two audio magazines or the Web equivalents - the review begins with many paragraphs of an excruciatingly dull navel-gazing psychodrama centering around the reviewer's appalling taste in music, two or three paragraphs lifted from the manufacturer's "white paper" and slightly rephrased, and a hastily-written conclusion that either damns with faint praise - the stab in the back - or puts the component in the "must-buy" category.

You can read several pages of this bilge before you get to any description of sound at all - and then, the description is cursory in the extreme. The meaningless weasel-words "accurate" or "euphonic" almost always make an appearance. If you're really lucky, you'll see wacko reviewer-speak words like "chocolate midrange" or other, equally disastrous metaphors. Discussion of the underlying technology is typically absent, or at best, a direct repetition of the half-truths or outright falsehoods of the marketing literature.

You can complain all you want about auto magazines, but this level of writing would never be acceptable in the auto world - the readers would laugh it out of existence. So what's happened to the readers in the audio world? Why do they tolerate this?

Thoughts?

[Chekhonte]

http://www.diyaudio.com/forums/showthread.php?postid=1310763#post1310763

Of particular interest:

Thoughts?

I understand where he is coming from but listening to music is purely a subjective experience. Top tier headphones and related gear all reproduce music very well and it's always very hard to articulate subtleties in any experience especially one that is as fleeting as listening to recorded music. The listener must convey how things are reproduced while taking into account his experience of the music. That's difficult terrain to navigate and is very different than describing how a car handles or what it's torque/weight ratio is.

[thrice]

I often wonder about this when I read comments like, "Well, I don't listen to graphs." Fuck me, graphs are a tool, nothing more, nothing less. If you're well informed about how to read the graph (measurements) then they will tell you something, not everything, but something. You can't build a house with just a hammer, you need a couple of different types of tools to accomplish that task... reasonably. A frequency response graph, or an impedance curve graph is helpful in that it gives you some information. We need to move past that shit and start looking at what we think sounds good and then look at the measurements and start to puzzle out why certain measurements are synonymous with gear we think sounds good. I'm sure there are folks out there that do that. I'm sure some of the more experienced members will say, "thrice, you didn't see this!" Good, I want to see it. We need to read that shit. Then we need to be "cool" about exposing people to that instead of being pretentious pricks to those who are not in the know.

On the other hand, measurements don't tell us the whole story (I'm sure this can be true to some degree with the automobile example used above as well.)

But beyond that little diatribe, I think the larger issue deals with what audio equipment is used for: playing music.

Music is very personal to people...sure driving might be personal as well, I'm not going to argue against that. How a well engineered car makes you feel as you drive it can equal the same satisfaction that one gets from listening to their favorite album or piece on their Hi-Fi. But when we deal with mechanical things it can be easier to deal with why something is better or worse. But what makes Beethoven sound good? Why are the Beatles so revered? We can talk about form, sound, scales, chord progressions, lyrics and all the mechanics of music, but there's always that extra little something that defies explanation.

It's that little something that mucks up the works, that and the fact that using live music, un-processed and electrified as a benchmark for comparison is getting rare. For the most part, most people experience music via recordings. Or perhaps it would be better to say that more people spend more time experiencing music via recordings than via live concerts. I know this crowd isn't quite the same as the non-audiophile crowd, but we do have a hobby focused around music re-production and not around live concerts. Sure we might love to go see live music and find that far more enjoyable, but we also gain a certain satisfaction with the fetish of collecting music recordings and buying music playback equipment.

So since we have that element we are not only looking for gear that conveys "the music" but we're looking for gear that also conveys the emotion that music typically brings up when we listen to it. So here's where it gets hairy. Try telling someone that their type of amp, or their tubes aren't as good as another type. People get defensive. Why? Because not only did you insult their toy (gear envy), but you also insulted their emotional experience.

I think this is part of the reason why people say, "trust your ears" or "you need to listen to something first." Opinions matter more than specs and measurements and technology, because an aesthetic quality comes into play. Is it a stretch to suggest that audio gear can elicit a wider range of emotions in the user than an automobile? a boat? a motorcycle? i don't think so. (of course that depends on whether or not you believe music can convey or elicit emotions, but let's not go there right now)

So then we bring that music to technology and that's were things collide. Some people swear by tubes. They swear you can get to the music better. I've had some pretty amazing moments listening with tube gear. I've also had some amazing moments listening with solid state gear too. Which is better? Which conveys the emotion and meaning behind the notes better? or at least conveys them as they happened or as the composer/performers intended them too? Oh yeah, what are the emotions and meaning being conveyed? Can we even begin to agree on that?

While I don't feel that terms like "chocolate midrange" or "silky highs" is helpful, how else do we explain it? If you're listening to a setup that's 1/2 the price and a better measuring setup and it sounds better to you...why is that? and vice versa?

We can come up with newer and better tools to make physical measurements and deal with a machine, but how do we measure emotional response? How do we measure if the emotion and meaning are truly captured by the mics? or other recording equipment? That's like asking someone to determine scientifically if Furtwangler captured the essence of a Beethoven symphony better than Ormandy or Levine...etc.

Of course that doesn't mean we should stop trying. By all means, no we shouldn't. But it incorporates a lot more than the author suggests. Aesthetics, theory, psychoacoustics, technology, physiology...etc.

So I think the initial author is not thinking fully about the topic at hand. Music is complex and difficult to qualify and quantify. It's not surprising that it's difficult to deal with the playback gear as well.

That's my take just off the cuff.

[humanflyz]

I think it's absolutely critical that the reviewer discloses his audio philosophy upfront and very explicitly before getting into whatever he's reviewing. I just don't see that happening all that often, and that's why I take these reviews with a big helping of salt. But the problem is that it takes a lot of experience listening to both live music and gear to be able to formulate a coherent and consistent audio philosophy, let alone articulate it to other people.

[Chekhonte]

i like natural burley tobacco better than aromatic tobacco, generally, but the Peterson 303 i bought yesterday is supposed to work very well with aromatic (it's designed to prevent an overly wet dottle), so i might have to try some nice aromatic cavendish once the pipe arrives, maybe some Nat Sherman #509.

When I smoked a pipe I loved black cavendish tobacco. The flavor and smell is so amazing. When I quit smoking it I used to buy an ounce every now and again and and put it in a dish over a candle as a type of incense.

[aerius]

I often wonder about this when I read comments like, "Well, I don't listen to graphs." Fuck me, graphs are a tool, nothing more, nothing less. If you're well informed about how to read the graph (measurements) then they will tell you something, not everything, but something. You can't build a house with just a hammer, you need a couple of different types of tools to accomplish that task... reasonably. A frequency response graph, or an impedance curve graph is helpful in that it gives you some information. We need to move past that shit and start looking at what we think sounds good and then look at the measurements and start to puzzle out why certain measurements are synonymous with gear we think sounds good. I'm sure there are folks out there that do that. I'm sure some of the more experienced members will say, "thrice, you didn't see this!" Good, I want to see it. We need to read that shit. Then we need to be "cool" about exposing people to that instead of being pretentious pricks to those who are not in the know.

On the other hand, measurements don't tell us the whole story (I'm sure this can be true to some degree with the automobile example used above as well.)

That I think is definitely a big issue, all too often we have reviews where the gear sounds great but measures like shit, or it measures decently and sounds like crap. There's a few rare components that both measure well and sound great. And that's about where it ends. Nobody outside of the DIY forums really looks at the whys behind it all to try and put together some of the pieces to the puzzles, the audio mags along with most of the online 'zines pretty much shrug it off and go "guess that's just the way it is". That is, if they don't go hardline with yet another subjectivist vs. objectivist strawman shitfest.

I find that sad since the general public never gets to learn anything. The mags gush on about the gear and they don't bother educating the readers on anything, UHF is the one exception since they do have a tech theory article in each issue. Using the car mag analogy, when GM, Honda, or Lotus comes out with some new engine tech, suspension design, or other new tech gadget, the car mags will usually have a column to explain what it is, what it does, and why it's good. Stuff like VANOS, Bluetec diesel, suspension linkage design, Miller cycle engines, CVT's and so forth all gets explained. Read a dozen or so issues of most car mags and you'll know most of this stuff.

Contrast that with audio mags, I could read the last 5 years of Stereophile & TAS and still not know the difference between a MOSFET, JFET, and a Bipolar transistor. I won't know what a Williamson or Dynaco circuit is, or what a Long Tailed Pair, Cathodyne, or Schmidt phase-splitter is and the advantages & disadvantages of each. I won't know why the slope of a crossover in a speaker is important, I'll know that 1st order crossovers are phase coherent and that's as far as it goes. I won't know the difference between R2R DAC chips and delta-sigma DACs, which is about as important of a difference as the one between gasoline & diesel engines as far as designing DAC circuitry is concerned. Nowhere do they explain anything about power supplies except to say "big transformers, big caps, Blackgates!", the reader is left to guess at what the fuck it all means.

It's almost as if the audio mags are deliberately trying to keep their readers stupid. Cause god forbid if your readers grew brains, they might figure out that something like say, the cRaptor is a flaming turd and a safety hazard to boot. And that would tend to give them second thoughts despite your gushing reviewer praise.

I feel that the audio mags have a duty to educate the readers so that they can make informed decisions. There should be tech articles, explaining for instance the pros & cons of soft dome vs. metal dome tweeters or the tech behind SE vs. P-P tube amps and speaker cabinet designs. UHF does have basic tech articles, they did one a little while back on power supplies and they're now running a series on room acoustics. AudioXpress which is mostly a DIY mag will of course get into all the nice calculations and talk about the theory & design issues, hell, they even broke out some calculus in one of their issues. You don't need to go that in-depth, but I'd like to see the mainstream audio mags cover some of that stuff on a fundamental level.

[humanflyz]

Yes, but how do you make that interesting or relevant to people who are no EE majors and have no idea what any of that means? I personally want to know, but my limits of understanding are pretty low since I'm a humanities student, and of course even among the knowledgeable types there are debates about the technical merits/disadvantages of certain designs, and after a point that debate becomes meaningless to non-engineers.

But in my opinion the pendulum has swung too far in the lowest common denominator direction judging by mainstream audiophile reviews. Some sort of balance is needed between getting all the technical stuff right and conveying an intangible experience.

[aerius]

Yes, but how do you make that interesting or relevant to people who are no EE majors and have no idea what any of that means? I personally want to know, but my limits of understanding are pretty low since I'm a humanities student, and of course even among the knowledgeable types there are debates about the technical merits/disadvantages of certain designs, and after a point that debate becomes meaningless to non-engineers.

I know it definitely can be done. The October 2007 issue of Hi-Fi News & Reviews has a nice article on measuring headphones, as well as how some of the measurements may correlate to how the headphones sound.

Run a tech article in basic easy to understand language in every issue, and have sidebars or even a whole page if needed in the reviews themselves to explain the unique features of the gear under review. For instance a tube amp review might have a sidebar on what fixed bias is, what its advantages are and why it's used, much the same way a car mag will have a blurb on say, Honda's VTEC technology.

After a year or 2 of reading, the readers should have a basic layman's understanding of the principles behind the designs, and you can start pointing them towards journal articles, research papers and so forth which discuss the issues in more depth.

[thrice]

Yeah I gotta say...I wish Stereophile integrated their measurements into their article more often. It's easy to gloss over them. It's certainly something that's lacking in the current popular Hi-Fi mags.