Percieving flaws as benefits

[Matthew J Poes]

I was talking with a reviewer friend of mine about some of the potential issues of listening to a sound system without any measurements. That we might perceive a deficit in the system as an enhancement without even realizing it.

Just as an example, it is possible that large dips in the bass at certain frequencies might give us the sense of tighter bass because there would be a lack of bass in the region that we perceive as boominess. Without a point of reference (a smooth response) and knowledge of how bass should sound, it may be difficult to know. Certainly when listening to the vast majority of 2 channel subwooferless systems, you would have major dips in the bass caused by things like SBIR.

My friend pointed out that a rise or prominence in certain parts of the treble region may give a speaker the perception of being more detailed and resolving, when in fact its simply inaccurate. Designers looking to capitalize on that may intentionally voice a speaker with this treble bump.

A dip in the midrange can give the sense of a deeper soundstage.

Phase inaccuracies between speakers can give a sense of a wider soundstage.

The list goes on in how psycho-acoustically we can trick our brains into hearing things we want to hear, even if they are not in fact accurate.

I'm curious what others think. Certainly when it comes to speakers, there is no definitive right. If you like it, it's right! If a speaker has the above mentioned deficiencies and someone likes it, that is certainly fine. What if this becomes a prominent part of professional reviews? What if they like or dislike a speaker in their room, but the issue is really an interaction with the room acoustically?

One thing I like about AV NIRVANA is that we have been publishing in room measurements of a speaker we are listening to. While the in room measurements may not tell us what the speaker measures like under anechoic conditions, it can tell the reader if the speaker was playing nice in the reviewers room.

 

[Todd Anderson]

This is an interesting topic, Matt. I think you know where I side on this. I'm much more of a hands-on impression (with in-room measurement) reviewer. Measuring a speaker outside can be tricky business with its own issues – and a concern is that a mis-measure or inaccurate measure that's implied to be comparable to an anechoic type measure could potentially mislead (either persuade or dissuade) a potential buyer, simply because someone may fixate on something in a graph.

The other side of the coin is that every speaker design is going to sound different in all of the unique environments it will eventually be deployed in, good measurements or not.

 

[Matthew J Poes]

This is an interesting topic, Matt. I think you know where I side on this. I'm much more of a hands-on impression (with in-room measurement) reviewer. Measuring a speaker outside can be tricky business with its own issues – and a concern is that a mis-measure or inaccurate measure that's implied to be comparable to an anechoic type measure could potentially mislead (either persuade or dissuade) a potential buyer, simply because someone may fixate on something in a graph.

The other side of the coin is that every speaker design is going to sound different in all of the unique environments it will eventually be deployed in, good measurements or not.

All fair points. I do think that outside measurements are not an absolute necessity in reviews and in fact should not be included in many if not most cases. However, as with any scientific test, speaker measurement protocols can be developed and vetted in order to ensure validity. Readers of this forum would not be aware, but you and I discussed this offline as I shared photos of my outdoor speaker testing (I believe your response was therapy was in order, haha!). I am currently working with someone who has an outdoor measurement protocol that was created with the head of a speaker development research department with a state of the art anechoic chamber. His method was vetted by the researcher including co-testing a known speaker and comparing the outdoor results to the anechoic chamber. Tweaks to the protocol were made until measurements agreed between the methods. We are now tweaking the protocol again as I am developing my own outdoor protocol, and in this case, Paul Barton (who has access to and decades of experience with an anechoic chamber) is providing the validation. Paul will be vetting my measurements of a known speaker and we will ensure that any anomalies are worked out. At that point, there really is no reason for an outdoor free space measurement to be untrustworthy. However, for readers of our reviews (or any review) to know this, they would need to understand that such validation was made.

I think the biggest issue with including such sophisticated information about a speaker under review is just being sure that it is explained thoroughly. In professional trade magazines, it is most common for the reviewer to simply test the speaker and provide results. No explanation. The equal of a "results" section of a scientific journal. Vance Dickenson's driver reviews for VoiceCoil magazine is a great example of this. For the trade or science community that is fine, but for most people that is wholly unsatisfying. I love the numbers, I love to see the graphs, but I also need explanations. What does the data mean? How might it correlate to my perceptions of the sound?

I also think that reviews (be it professional or personal) should be done blind to the test data. I do think that seeing the test data before listening just biases you for or against the speaker. Since outdoor free space measurements are so involved, I would always need to do them long after having listened to a speaker and drawing subjective impressions. I would likely take indoor measurements in room just to be sure they are setup correctly and optimally placed, but in that case, its mostly bass that is of concern. Speakers measure VERY differently in room than they do in free space. Things that look horrific in free space may nearly disappear once in a room.

but alas, we've drifted from the intent of this post! I really was more wondering if actual flaws (not in the speaker design necessarily) that can play into psycho-acoustics could lead to misperceptions of the benefits of a given speaker, and what people think of this.

 

[Todd Anderson]

Good point about having a pre-review bias. I agree 100%.

As for your intended discussion, what matters most is how the speaker sounds to the listener (deficiencies or not). When it comes to professional reviews, there's certainly an aspect of the room/speaker positioning that come into play. But a review can set a decent baseline expectation of perceived quality and overall "generalized" performance that readers can use to decide if a speaker is worth exploring (or not).

Just MHO of course... what's your take?

 

[Matthew J Poes]

Good point about having a pre-review bias. I agree 100%.

As for your intended discussion, what matters most is how the speaker sounds to the listener (deficiencies or not). When it comes to professional reviews, there's certainly an aspect of the room/speaker positioning that come into play. But a review can set a decent baseline expectation of perceived quality and overall "generalized" performance that readers can use to decide if a speaker is worth exploring (or not).

Just MHO of course... what's your take?

I do agree with what you are saying, but I guess I remain concerned that sometimes people like things that maybe they shouldn't. I know, that's is unfair. It's all subjective, if you like it you like it? At the same time, maybe we like things because we got used to it, and again, because the "flaw" played into a psycho-acoustic phenomena. Harman has shown in some of their preference research that it may take a proper point of reference and training to be able to appreciate when something is "right."

I think the best example is a bright speaker. I don't think we can argue that a bright speaker is right, its bright. It is not neutral. Yet a lot of people prefer this. It could be their hearing isn't great and the brightness compensates. Everyone suffers some degree of hearing loss as they age. It could also be that the brightness cuts through the extreme bass the system has. I've found this VERY true of bass heads! it's hard for a neutral speaker to sound like it is keeping up when everything below 100hz is coming from four 18" subwoofers (or more) and 5000 watts of power (or more!). Like I said earlier, who is to say that's wrong? If that is what you like, this is all about entertainment, you should have bright speakers.

 

[Todd Anderson]

I like brighter-ish speakers.

You calling me old?

And I do like extreme bass....

Wait a second.

I actually think bass is a good example. I like to run my system a little on the hot side, which likely takes it slightly away from what some would consider artist's intent. But, at the end of a day, I think speaker reviews (in particular) are best used as a guide, not ultimate gospel... in many cases.

 

[Matthew J Poes]

I like brighter-ish speakers.

You calling me old?

And I do like extreme bass....

Wait a second.

I actually think bass is a good example. I like to run my system a little on the hot side, which likely takes it slightly away from what some would consider artist's intent. But, at the end of a day, I think speaker reviews (in particular) are best used as a guide, not ultimate gospel... in many cases.

Haha, hey I did say that there is nothing wrong with liking it, that this is all about entertainment.

I think the "issue" comes in more with a statement like "XYZ speaker was more resolving than ABC speaker" when in fact, XYZ speaker is simply bright. It gave the psycho-acoustic impression of more detail by highlighting aspects of the music that we perceive as the detail.

In your case, you make very clear your preferences and your in room measurements show the room curve you like. It is then easy for a reader to look at your impressions and make decisions from there.

I'm quite the opposite, I find excess "detail" in the form of brightness irritating and prefer a system that is on the warm side of neutral. When I design speakers myself or tune a system (for myself) I put in place a 3db per octave roll-off above 1khz.

I like extreme bass for movies, but not for music. I tend to prefer roughly a 3-6db per octave rise in the bass starting at about 150hz. I also find myself fiddling with the bass all the time, having even saved multiple EQ curves on my sub amps because of what Floyd Toole calls the "Circle of confusion." Much music is mastered such that it is not properly balanced simply because the mastering system itself was not neutral.

Here is an example of a bookshelf speaker crossover I mocked up with my preferred response shape, showing 15 degrees (the listening axis) and 45 degrees:

 

[Tony V.]

Look at how may people apply a house curve to their speakers or use loudness to listen to music. The missions I have up in my livingroom are warm sounding speakers and are nice for the awkward livingroom space I have them in. they did not lend themselves well to the theater room when I had them down there compared to the EVs I currently use. however even looking at my REW graph of the EVs they fall off at above 16K fairly quickly but dont sound at all like they lack the detail at all listening to music or movies and I never suffer from ear fatigue.

 

[Matthew J Poes]

Look at how may people apply a house curve to their speakers or use loudness to listen to music. The missions I have up in my livingroom are warm sounding speakers and are nice for the awkward livingroom space I have them in. they did not lend themselves well to the theater room when I had them down there compared to the EVs I currently use. however even looking at my REW graph of the EVs they fall off at above 16K fairly quickly but dont sound at all like they lack the detail at all listening to music or movies and I never suffer from ear fatigue.

My speakers fall off at 18khz. The dome in the CD breaks up around here and creates a deep null, effectively squashing output much past 20khz. Most adult males have little hearing above 15khz, especially up past 18khz. Our sense of detail actually comes from much lower treble frequencies. Speakers that roll off aren't a huge concern to me. I'm interesting in playing around with speakers that have a response out to say 50khz and high sampling rate music to see, given some of the research into this, but in general I don't see this as a flaw. More like omission.

Even house curves don't bother me. I suppose applying a house curve to a speaker under review could be seen as dishonest as the speaker won't then sound the same tonally to anyone else who doesn't apply the house curve. At the same time, many experts seem to feel that the "house curve" shouldn't be built into the speaker and should be applied in the preamplifier/processor in the room. Since I design speakers for me and nobody else I actually do include the house curve I like in the speaker design itself.

 

[Tony V.]

I can understand headphone listeners wanting clarity up above 20kHz but for in room I dont know if its of any benefit given rooms are never perfect acoustically and those frequencies get mashed up fair bit unless you sit 3ft away from the speakers.

 

[Matthew J Poes]

I want to preface this by saying I'm not picking on these speakers. They happeb to make good examples for the point I was trying to make.

My speakers, a speaker I refer to as having a pleasant neutral but on the warm side presentation.

Response has a downward tilt above 10khz, though it should be noted that the wider dispersion up there means it isn't as rolled off as it looks. The DI on this speaker is nearly as good as it gets.


A small JBL Monitor with excellent performance. By comparison this speaker sounds bright to me. The treble at 10khz is actually louder than midrange at 1khz or midbass at 100hz. You can also see that while the treble appears to extend farther out, it has very narrow directivity (beaming) which causes a non flat DI.

Now let's look at some speakers with some issues:

Ascend Acoustics CBM-170SE is show above. This speaker is nearly omnidirectional up to 15khz but then the treble beams quite a bit. Not horrible but it can effect imaging to have such dispersion. It also lacks baffle step and sounds thin. This is actually a good speaker if your goal is such a dispersion pattern.


Dynaudio Xeo 3: Clearlt needs to be listened to a bit off axis. This speaker has rolled off highs and an elevated midrange. I've heard it and thought it gave a false sense of detail and created a superficially forward sound stage.


Polk Audio 5JR:. This speaker has rolled off highs and a peak at 10khz. It also has uneven dispersion and an ugly DI. I've not heard this particular speaker but have heard related models. They add sibilance and edginess to the music. Not horrible but it's there. I think some percieve that as detail.

 

[AudioThesis]

The 5JR uses the SL2000 tweeter which is notoriously bad and hated even among the Polk enthusiasts. The RD 0194 and RD0198 is a current, drop-in replacement that transforms these older monitors from Polk.

As far as measurements go, they are but one tool because tone is but one tool. I don't take measurements when I demo a speaker, but I have a basis of hundreds of systems now and have trained my ear fairly well. Things I listen for to determine the quality of a speaker fall more in line with subjective items instead of objective - things that can't be measured, or at least measured well. There are too many variables that contribute to make it a critical part of the review, but like you pointed out, it could validate some observations.

Reviewing is a tricky art and nobody really masters it to the point that they satisfy everyone. Some go heavy on objective measurements, others go heavy on subjective results, and others have a decent mix of them. Whoever your audience is determines how you reach them. Not to get off topic, but I've found the reviewers I buy into have similar tastes in gear that I do - ie, they LOVE single ended amplifiers and can explain why they do.

 

[Todd Anderson]

The 5JR uses the SL2000 tweeter which is notoriously bad and hated even among the Polk enthusiasts. The RD 0194 and RD0198 is a current, drop-in replacement that transforms these older monitors from Polk.

As far as measurements go, they are but one tool because tone is but one tool. I don't take measurements when I demo a speaker, but I have a basis of hundreds of systems now and have trained my ear fairly well. Things I listen for to determine the quality of a speaker fall more in line with subjective items instead of objective - things that can't be measured, or at least measured well. There are too many variables that contribute to make it a critical part of the review, but like you pointed out, it could validate some observations.

Reviewing is a tricky art and nobody really masters it to the point that they satisfy everyone. Some go heavy on objective measurements, others go heavy on subjective results, and others have a decent mix of them. Whoever your audience is determines how you reach them. Not to get off topic, but I've found the reviewers I buy into have similar tastes in gear that I do - ie, they LOVE single ended amplifiers and can explain why they do.

I like your sentiment: It really does boil down to the audience. There's the end-result "how does it perform crowd"... which is nearly opposite to the "how does it work" crowd.

 

[Matthew J Poes]

The 5JR uses the SL2000 tweeter which is notoriously bad and hated even among the Polk enthusiasts. The RD 0194 and RD0198 is a current, drop-in replacement that transforms these older monitors from Polk.

As far as measurements go, they are but one tool because tone is but one tool. I don't take measurements when I demo a speaker, but I have a basis of hundreds of systems now and have trained my ear fairly well. Things I listen for to determine the quality of a speaker fall more in line with subjective items instead of objective - things that can't be measured, or at least measured well. There are too many variables that contribute to make it a critical part of the review, but like you pointed out, it could validate some observations.

Reviewing is a tricky art and nobody really masters it to the point that they satisfy everyone. Some go heavy on objective measurements, others go heavy on subjective results, and others have a decent mix of them. Whoever your audience is determines how you reach them. Not to get off topic, but I've found the reviewers I buy into have similar tastes in gear that I do - ie, they LOVE single ended amplifiers and can explain why they do.

Thanks Skip, always thoughtful perspectives.

A few things to keep in mind from what I presented above:

• The polk was not presented to pick on Polk but just to show what a speaker with obvious measurement flaws looks like and how it can impact the sound. It measures bad and apparently sounds bad (since you mention even POLK aficionados don't like them)
• These measurements are not like any other measurement you see floating around, nor are they something the average person can take on their own. These are full polar measurements taken in anechoic conditions. Frequency response data doesn't just show tonal balance or tonal anomalies, and when you create more comprehensive data like this, they can tell you nearly everything you can want to know. We can argue this point, but of course, its just a subjectivist argument, one in which there is never a definitive answer, just a perspective. However, from this data you see not only tonal balance but clues as to how it will soundstage in a room, how it will sound tonally, how it will sound as you move your head, how the image will stay stable, etc. If I added in additional information for distortion at various output levels, transient response data, etc. we could begin to look at other factors too. It's important to remember that if a speaker's transient response is too bad, so is the frequency response, it is a iron law of physics because the speaker is minimum phase. A speaker can't ring if there isn't an excess of energy in the response in the first place (a rise). Many of the subjective terms we use can and have been tied to measurements and so it isn't so much, can measurements tell us, but understanding how the measurements tell us about the perceived sound.
• The point of all this was not to pit a subjective and objective view against each other. To me that is utter nonsense. In my own field we do not consider these mutually exclusive views and in fact, we know for an absolute fact we cannot measure everything that humans experience. We use what is called mixed methods, but our subjective research known as qualitative research relies on very regimented protocols to enhance the rigor and reliability of such work. This is a very important point, people are extremely fallible when it comes to perceptions. We perceive the world around us incorrectly. We see good things as bad, bad things as good. We filter the information coming into our brains to fit a schema we have created. This is a well established science and fully passes the common sense test. Objective methods that rely on quantitative methods do not suffer these biases when done properly (in fact they are always nothing but numbers, any bias comes from the subjective human). Because we are so easily fooled, I think its important to learn this, to understand this, and to seek to avoid this in reviews. Be it choosing your personal next speaker or reviewing professionally for others.
• Finally I'll just say, I think you know me well enough now to know that while I am more of a quantitative person, I listen with my ears first and my brain second. I do not rely on measurements to decide on sound. In fact, I feel strongly that to be objective and unbiased in my qualitative review of a product, I must not look at the quantitative data. I do use measurements in setup due to bass (which is just a placement and room acoustics issue) but I intentionally avoid looking at anything above the bass so as not to bias my views. The same principles are followed in most sciences, the researcher must remain blind to the results of a separate test to avoid confirmation bias (using a preconceived notion formed by other experiences to bias your interpretation of the information in front of you, seeing things the way you want to see them). I believe that quantitative and qualitative methods of inquiry cannot be separated when dealing with human perception, we cannot measure accurately what we perceive, but we also cannot remove bias nor be an accurate tool of measurement ourselves. The two methods compliment each other and act as checks on the conclusions we would draw from either.
• I also like SET amps quite a bit, yet know full well they measure badly. I also have studied them enough to understand why that is. In fact measurements DO tell us quite a bit about why the measure worse but sound better, and that goes back to my point. Numbers are numbers, it's the human that is biased. THD and IMD are poor correlates of sound quality and this is well established. Thresholds of audibility and how we perceive distortion is also poorly researched, far worse than people like to admit. Research conducted as far back as the 70's showed that in blind trials there was a clear preference for tube and especially SET amps in comparisons to solid state and they could predict why through information on the transfer function, especially looking at the distortion profile of that transfer function. Tube amps naturally gave a preferable response shape mixed with distortions we perceive as pleasant, with an ability to better mask or even eliminate distortions we find objectionable (and these distortions we find objectionable are at very low levels). Modern amp designs rely on a method that increase objectionable distortion to reduce less objectionable distortion because it lowers the THD and IMD values. So when someone tries to argue that tube amps suck because they are objectively bad, they fail to understand the actual science. It's an issue of compromises, and it seems likely that some of us would place greater emphasis on things like the sound of the distortion than we would on things like greater output or lower noise floor (even I who love a low noise floor can easily hear through the noise, but I cannot ignore the distortion). Add to that the fact that tube amps soft clip and that nearly all amps will clip in use, and it becomes quite clear why tube amps, especially SET tube amps are preferred by some (SET have the lowest switching and higher order distortions of all amp types, all else being equal). All of my guitar amps are tube and at one time my main stereo amps were tube, including an 805 based SET, but I was forced to make a choice and ultimately sold the SET due to how it heated up my room and its danger for my wife and child (who equally have an inability to avoid touching the exposed high voltage parts).

 

[Matthew J Poes]

I like your sentiment: It really does boil down to the audience. There's the end-result "how does it perform crowd"... which is nearly opposite to the "how does it work" crowd.

As I mentioned in my previous post, how something works and how it sounds shouldn't be separate crowds. Designing good sounding electronics is a science, no denying that. How it sounds to us is entertainment, it is not a science. Therefor we must use a very specialized method of scientific inquiry to understand how something works as a means to predict how it will sound.

My belief is that humans are poor scientific instruments, we are simply too biased, but for casual reviewing this doesn't matter. I believe that it is theoretically possible to measure everything we perceive accurately and develop models that tell us if any one person would like the sound of a product or not. However, I also believe that we are not there yet. There is no science to tell us how a speaker pair should measure in room to have a good soundstage (even though it is in fact possible to measure soundstaging). There is exceedingly poor data to show how we perceive distortions of different types and how that impacts our perception of sound quality. We have no way to model and predict these qualities and must then rely on our own ears instead. When it comes to the kind of information we can get from spin-o-rama data as I presented above, I believe that science is solid at this point. Harman (among others) have so extensively tested this that it is not easy to develop a computer model of a speaker and know how people will perceive it (desirable or not) in real life, but only based on what that data tells us. As I understand it, When this work began and Floyd Toole developed the approach with his team, the feeling was that this was the most important aspects of sound quality. Nothing else mattered enough to be a part of the auralazation work. In talking to other experts of equal caliber, what I've heard is that there is great debate still on if Harman's work really captures everything that matters.

Another major problem with perception research is that its inference based using a method of deduction that concludes based on probabilities. This is in fact my field of study and my area of interest right now is specifically understanding why such methods, which rely on interpretation across masses based on averages, doesn't do a better job capturing the views of most/all people. With medical treatments, when we say a treatment works, we aren't saying it works for everyone. We are saying that on average the probability that it will work for a randomly selected person is better than chance (even this is a slightly inaccurate over-simplification of what the stats tell us). In practice this means that many people will not be like the average proto-person we tested, and in fact, the stats don't even tell us that most people are like this "average" since the actual effect can be quite small, even if significant. This all maps well onto audio perceptions. The research that groups like Harman did show what an average person with good hearing will like, but not what everyone will like. That means there is a lot of objective wiggle room to debate what is accurate sound for any given person, since we developed a science that says accurate is based on preference. The only thing we don't apply such subjectivity to is the smoothness of the frequency response and directivity, where it instead follows logically that a lack of bad undulations in the response is a good thing, more true to the source, as they say. In fact, even this may not be true if for no other reason as we may not be able to hear it.

So in the end I say that I believe both tools at our disposal, our qualitative perspectives and our quantitative measures are flawed. Neither gives us all the information and I think it is important to use both in the least biased way to draw the best conclusions. Be it for our personal fun or the betterment of public understanding. What I don't like is the misuse of either tool to create misinformation.

 

[AudioThesis]

Perception is one thing - training is another. Ask Joe Shmoe if he can call out long distances and you're going to get varied results. Focus on guys who go to the range frequently and you're going to get a much tighter bullseye if you will.

I only spoke about the 5JR so as not to blame it as being a bad speaker given the replacement tweeters available. The graph is fine and represents issues found in all Polks using the dreaded SL2000 tweeter.

I feel like you were defensive in your post and I hope that wasn't the case. I had no intent of attacking you in any way, but offering my perspective on the topic. I do think your approach of using qualitative measures first and then introducing the quantitative aspects is the proper way.

 

[Matthew J Poes]

Perception is one thing - training is another. Ask Joe Shmoe if he can call out long distances and you're going to get varied results. Focus on guys who go to the range frequently and you're going to get a much tighter bullseye if you will.

I only spoke about the 5JR so as not to blame it as being a bad speaker given the replacement tweeters available. The graph is fine and represents issues found in all Polks using the dreaded SL2000 tweeter.

I feel like you were defensive in your post and I hope that wasn't the case. I had no intent of attacking you in any way, but offering my perspective on the topic. I do think your approach of using qualitative measures first and then introducing the quantitative aspects is the proper way.

Oh gosh I must be getting crotchety in my 36 years of old age. I didn’t mean to be defensive. I was trying to be informative if anything. That measurements can tell us quite a bit. Not everything of course, but quite a bit.

I really don’t like the subjectivist vs. objectivist camp views. There is real flaws in such perspectives as each is missing something important from the other. What I’ve always liked about folks like John atkinson is the ability to balance both fairly well.

I’ve said this to you before but I find it sad that I have to somehow pick a side. That if I present measurements and educate my readers of the science, that I can’t talk about things I hear but can’t measure so easily. If I think a particular cable, isolation stand, or capacitor sounds better I’ve betrayed the science I hold dear.