"Spectral Incoherence" or "The Cocktail Party Effect" - What is it?

[Bruce Swedien]

Think of this for a minute....

 

In his book "Spatial Hearing" Blauert says our hearing system deals with the complexity of the human hearing process by employing something known as "spectral incoherence".

 

If you were to analyze a sound as it entered each ear canal you would find the expected differences in frequency and phase response between the two ears. But this really doesn't tell us all that much.

 

When we look at a sound as two separate sonic events,(time and frequency) you will see there are similarities as well as differences (i.e. 50% correlation) which allow the two ears to work together to form a spatial impression, not neccessarily directional because we are dealing with multiple events not just a single sound source.

 

This quality, or I really prefer to call it an ability in our hearing, I think plays a major role in what I like to call the "cocktail party effect". This means, that a person with two good ears can pick out one conversation from many, while a person with only one working ear cannot. Along that same line of thought, have you ever noticed how, when you monophonically record a person speaking in a reverberant room, (Like on a small mono cassette recorder) the recording sounds highly reverberant and essentially unintelligible. Yet when you are in that very same room with that person speaking, you can understand every word. This is, of course, our binaural hearing system in action. Our two good ears connected to our brain are able to separate the direct sound from the refelected sound and give us a highly intelligible sound image. A monophonic microphone and tape recorder cannot.

 

What Im really talking about is that when we listen, we hear sounds with two ears, and thus utilize our binaural hearing. The sound waves reaching the two ears will usually not be identical. For low frequency sounds, of long wavelength(compared to the size of the head), there will generally be a phase difference due to the slightly different arrival times of the sounds at the two ears. For high frequency sounds, of short wavelength, there will also be an intensity difference due to the fact that one ear is farther from the source of the sound and is also in the sound 'shadow' of the head. However, despite these differences, we usually hear only one sound. This I call binaural 'fusion'. In the processing of these sounds, however, the brain utilizes these differences to enable to tell us what direction the sound is coming from. This process is called "Localization". "Localization" is, in reality, the basis for the stereophonic effect in music recording.

 

The real trick is to figure out how we can use all this esoteric and scientific knowledge and make it help us in recording music...

 

With love from an aging young Swedish Windbag!!!

[Jimbroni]

I'm using the assumption that the word spectral applies to frequency domain, and temporal applies time domain. What I find interesting here is the Frequency domain portion of the question. What impact on perception does the spectral portion give us. I think its pretty well known to acousticians that our ability to percieve spatial position or directivity of a sound is based on the time lag from one ear to the other. Were not so good with up and down or the z axis. Now onto the spectral part, Have we a dictionary of tonal info in our heads just waiting to be unlocked? Is there a way to mess with spectral effects to emulate the tone of a wood floor versus glass in say a reverb module?

[Angelo Clematide]

Excellent BRUCE, outstanding, gratulation and thanks

 

i'm very impressed with the ease you write your texts on complex matters.

 

I compiled some acoustic phenomena/phenomenons links for audio engineers in the

David Frangioni: Studio Tech

forum here.

[miki]

Mr. Swedien> Good sir, you are out of my league as I am not a recording engineer nor technician, but I read what you write, follow fairly closely and I learn. Please pardon me for posing my thoughts.

I get the complexities of what you're talking about however, like most things of the senses, doesn't it boil down to how each person's brain interprets the information which is being sent?

<< The real trick is to figure out how we can use all this esoteric and scientific knowledge and make it help us in recording music... >>

Given the information above, doesn't it then stand to reason that each person's perception is different therefore, how one engineer/technician perceives/interprets the sound may be significantly different than how another perceives it? Isn't the 'real trick' then more of how each individual can better perceive the sounds rendered in order to produce better recordings and wouldn't that be extremely subjective rather than objective science?

[Angelo Clematide]

Oh Miki Miki, here you come with a great question...

[miki]

Yikes, other scary thoughts/questions...

Are we trying to eliminate the human factor so that personal perception is no longer necessary? Shove all the sound into an artificially intelligent gizmo/DAW/mixer/blah... and let it do all the perceptual interpretation for us?

Science of sound='evolving' through elimination of the detestable (in some cases)'human factor' as needed?

I'm no engineer but that can't be a good thing from a recording engineer career standpoint.

[Jimbroni]

Isn't the 'real trick' then more of how each individual can better perceive the sounds rendered in order to produce better recordings and wouldn't that be extremely subjective rather than objective science?

I dunno thats kinda like saying hey I'm buddhist and you're a christian we disagree on some things, so you and I should change how we percieve things to line up somewhere in the middle. That just don't jive. It certainly would be a trick, but both of us would be living a lie.

 

Going the other direction as Bruce suggests, you can use emperical data to deal with the subjective parts. In fact the meaning of the word loudness, can be defined mathematically with a scale called sones. This scale was developed by the phone companies in the early days, to put a number on this thing loudness. It was based purely on statistical data. In the automotive community subjective acoustics are used all the time with a piece of software called Artemis. Jaguar developed a generic term called "refined power". which they thought was a good description of how they thought their cars should sound while driving. They did a bunch of spectral measurements on all of their cars. Using the statistical parts of artemis, they plotted a spider chart. What they found was every automotive company they benchmarked, somehow lumped together. All of GM's vehicles occupied a specific region the graph, all of BMW's another, etc. This suggested there was some sort of sonic signature that each manufacturer possessed. By analysing the data further they were able to figure out that the common ground of Jaguar vehicles, was a certain amount of energy in specific lower frequency bands along with a smooth slope going downwards with increasing frequency. They surmised this meant the lowend content is where the sense of "power" came from, and the smoothness of the spectral slope is where the sense of "refinement" came from. Could you imagine the possibilities if advanced recording engineers starting developing these sorts of spider charts to develop sonic signature of certain effects, rooms, materials, and instruments?

[Angelo Clematide]

great

[paully]

One related side topic. As you age, and especially if you have been exposed to loud sounds(as lots of musicians have been), you lose this ability to zero in on a specific conversation in the presence of ambient noise. Probably why you hear the expression 'HUH?' alot from older folks. Good reason to turn it down.

[Ben One]

I'm very fortunate to work with members of the Acoustical Society of America (Jens Blauert is one of their stars)--you are asking a very interesting question and you are on the right track.

 

There are three main factors in localizing a sound source--interaural level difference (the difference in the sound level reaching two ears), interaural time difference (the difference in the time when the sound reaches the two ears), and last but not least, what has become hugely appreciated over the last few years is the "anatomical transfer function"--the difference in how the sound is scattered by the listener's outer ears, head, shoulders, and upper torso.

 

When I was going to elementary school, my sixth grade teacher told me that the outer ear was useless from an evolutionary point of view and might not be needed further along our evolutionary line. Well, she was wrong. Researchers are finding that the way sound gets scattered from the outer ear (and other parts of the body) is extremely important in determining where a sound came from and localizing a stereo image.

 

This effect has been exploited very recently in products like the Niro single surround speaker, in which one speaker can effectively simulate surround sound, by taking advantage of how sound from a given angle bounces off differently from different sides of the body.

 

Anyway, what I've told you is explained in more technical detail in a Physics Today article by another acoustician, William Hartmann of Michigan State: http://www.aip.org/pt/vol-52/iss-11/locsound.html

 

The author has also made an astonishing recent discovery that individuals may have earedness--a preferred ear--in addition to handedness. Though the effect is not as important as handedness, it's pretty cool:

http://www.acoustics.org/press/147th/press_release.html#t02

The writeup doesn't have the link to the soundfile, but you can find it here (it only works with stereo headphones):

http://www.parmly.luc.edu/parmly/huggins.html

 

I may have not answered your original question, but I hope this may have provided additional food for thought!

 

Ben

[Kendrix]

Originally posted by BenOne:

I'm very fortunate to work with members of the Acoustical Society of America (Jens Blauert is one of their stars)--you are asking a very interesting question and you are on the right track.

 

There are three main factors in localizing a sound source--interaural level difference (the difference in the sound level reaching two ears), interaural time difference (the difference in the time when the sound reaches the two ears), and last but not least, what has become hugely appreciated over the last few years is the "anatomical transfer function"--the difference in how the sound is scattered by the listener's outer ears, head, shoulders, and upper torso.

 

When I was going to elementary school, my sixth grade teacher told me that the outer ear was useless from an evolutionary point of view and might not be needed further along our evolutionary line. Well, she was wrong. Researchers are finding that the way sound gets scattered from the outer ear (and other parts of the body) is extremely important in determining where a sound came from and localizing a stereo image.

 

This effect has been exploited very recently in products like the Niro single surround speaker, in which one speaker can effectively simulate surround sound, by taking advantage of how sound from a given angle bounces off differently from different sides of the body.

 

Anyway, what I've told you is explained in more technical detail in a Physics Today article by another acoustician, William Hartmann of Michigan State: http://www.aip.org/pt/vol-52/iss-11/locsound.html

 

The author has also made an astonishing recent discovery that individuals may have earedness--a preferred ear--in addition to handedness. Though the effect is not as important as handedness, it's pretty cool:

http://www.acoustics.org/press/147th/press_release.html#t02

The writeup doesn't have the link to the soundfile, but you can find it here (it only works with stereo headphones):

http://www.parmly.luc.edu/parmly/huggins.html

 

I may have not answered your original question, but I hope this may have provided additional food for thought!

 

Ben

Ben,

Interesting article- Thanks fo the link.

I havent read Physics Today since I was just out of college (physics major here).

I take it thats your field as well.

 

When it comes to understanding exactly what goes on in the human nervous system things get pretty murky. Making the connections between the empirical observations reported here and a model for whats really goin on in our heads that allows us to interpret these subtle differential signals from our ears seems a long way off.

 

Thank God we dont need to understand all this to enjoy good sound.

[KenElevenShadows]

Originally posted by miki:

Yikes, other scary thoughts/questions...

Are we trying to eliminate the human factor so that personal perception is no longer necessary? Shove all the sound into an artificially intelligent gizmo/DAW/mixer/blah... and let it do all the perceptual interpretation for us?

Science of sound='evolving' through elimination of the detestable (in some cases)'human factor' as needed?

This and other discussions are an attempt to understand personal perception, how the mind/ear works, so that we can create better recordings that enhance the listener's appreciation of, well, whatever's being heard.

 

The listener, in the end, is always ultimately responsible for the intepretation.

[KenElevenShadows]

Originally posted by miki:

Given the information above, doesn't it then stand to reason that each person's perception is different therefore, how one engineer/technician perceives/interprets the sound may be significantly different than how another perceives it? Isn't the 'real trick' then more of how each individual can better perceive the sounds rendered in order to produce better recordings and wouldn't that be extremely subjective rather than objective science?

Of course it is subjective. It's an art (and a science). But there are fortunately enough commonalities that we can speak a common lingo and often understand each other (just like many of us can agree that the color "red" looks a certain way, but not *everyone* perceives it that way - see my coments on synesthesia in another thread).

 

But there are still bdifferences, as you point out. And the differences are the reason why you may feel that a mix is really great while I feel that it blows chunks, or vice versa. Or that you might feel that a Telefunken ELAM 251 sounds better on a vocal than a U47. Or that it might sound good on a set of monitors but lousy in your car. Or not. We acknowledge these differences, while still trying to achieve the best recording possible that exudes the creativity, emotion, or whatever that initially went into the song/recording/whatever.

 

It's all about making choices, often subjective, artistic, creative choices.

 

It's NOT about someone dictating terms to everyone else. I doubt that's even possible.

[KenElevenShadows]

Originally posted by Bruce Swedien:

This I call binaural 'fusion'. In the processing of these sounds, however, the brain utilizes these differences to enable to tell us what direction the sound is coming from. This process is called "Localization". "Localization" is, in reality, the basis for the stereophonic effect in music recording.

Absolutely.

 

I should mention here that in addition to the actual ears, our binaural matrix, errr, um, brain, is the one that largely interprets the incoming information. And what it does is incredibly complex. It interprets the force of a sound coming from an area that is blocked (which decreases intensity), and interprets the location of sound despite reflections (although obviously it can be deceived). And we of course have the ability to literally focus on a particular sound, excluding others. And as Bruce mentioned, hearing loss can interfere with this ability to differentiate sounds. And so can aphasia.

 

For interest's sake, auditory/psychological studies in the past several years have indicated that binaural cues are used primarily for determining azimuth while monaural cues are, for the most part, used for determining elevation. Although this bodes somewhat well for someone who has monaural hearing (one ear working, in other words), the problem with this is that we typically experience sounds on the horizontal plane, not the vertical plane. This is part of the challenge if one has only ONE good working ear.

[Ben One]

While our perception and enjoyment of sound is ultimately highly individual and subjective, and it is tempting to let it remain fully mysterious, it is useful to know what scientific information is out there. There is more known about these topics than one may initially realize. And not to worry: while the science information is fascinating, it will never take the fun out of our ability to perceive sounds in individual ways.

 

There are at least a couple of different topics that we're discussing here:

 

--our ability to tell where a sound is coming from (localization)--this ultimately helps us form a stereo image

 

--our ability to separate multiple sound sources into individually identifiable elements--for example, Bruce's cocktail party example in which a listener can hear one voice amidst a background of many voices. Another incredible example is when we go to see a symphony orchestra. At any given instant, just a single composite sound wave is hitting each ear, yet we are able to separate it out into all of the beautiful instruments of an orchestra.

 

As for the second topic, evidence is mounting that our ability to separate sound is not purely a sensory phenomenon--it is not something that can be explained purely by the differences in the waves hitting each ears. There's something in the central nervous system--the brain--that processes these sounds and allows us to separate them.

 

Anyway, since I am not fully knowledgeable about what the current conclusions are regarding the Cocktail effect and the general ability to separate multiple sources, let me just defer to a few more links on these topics. The first is a 1992 review of the Cocktail Party effect, the second is an interesting sound experiment in unmixing sound sources, and the third shows some of the latest research in trying to figure out how we separate distinct sound sources at the higher levels of the brain.

http://www.media.mit.edu/speech/papers/1992/arons_AVIOSJ92_cocktail_party_effect.pdf

http://www.auditory.org/postings/1998/439.html

http://www.acoustics.org/press/147th/sinex_ll2.htm

 

Ben

[KenElevenShadows]

Originally posted by BenOne:

As for the second topic, evidence is mounting that our ability to separate sound is not purely a sensory phenomenon--it is not something that can be explained purely by the differences in the waves hitting each ears. There's something in the central nervous system--the brain--that processes these sounds and allows us to separate them.

Yes. That's the "Binary Matrix" that I jokingly referred to at work. And evidence has been mounting for a while. When I took my physiology courses in the '80s, this was being already discussed.

 

Thanks for the links! I love stuff like this.

[lrossmusic]

I sometimes wonder what a piece of music may sound like to someone else. Since we have no way of ever really knowing or listening through someone else's ears, I guess we are all kind of living in our own little sonic worlds. Wouldn't it be interesting if we could somehow hear what say a trombone sounded like to someone else the same way we can listen to a recording of it through different speakers or headphones. I know sometimes when I hear a recording of myself speaking it sounds a little strange for a minute whereas another person speaking on the same recording sounds to me the way they normally do.

[Jimbroni]

Thats because you always hear your friend through the air. As compared to yourself, which is combination of vibration transmitted through the bones in your head and airborne sound. When you hear the tape, you're hearing yourself through the air only. This in itself indicates the physical shape of your body has an impact on how things sound to you. Not the mention the other biological factors you guys have discussed, which is very interesting BTW. Thats why I suggested earlier that focusing on studying the effects of external objects, (instruments, walls, materials) might be a fruitful exercise, because those things are more reliable as reference points. Though there can be a lot of potential reference points. Maybe another study similar to the loudness study that I mentioned earlier could be done. You would take generic refence sounds found in music industry in a jury room, come up with a rank order system. Comparing different terms that we use around to describe sound with. We all do it here and there. Warmth is down in 60hz, but so is muddiness. So take a bass drum, ask people to rank it 1 to 10 in terms of warmth. Then scramble, do the same but in terms of muddiness. While doing all this take spectrum measurements, so later you can use the data to correlate back to the subjective ranking. Boy do I feel like geek all of a sudden. But now we could potentially have a link between the objective and subjective. Which could be used for a mathemical basis for adding more accurate timbral effects to digital modeling. Of course none of this helps the average Recording Engineer. Unless, you developed a realime analyzer plugin with the added subjective descriptors, or realtime spider charting.

 

 

Thanks for the links, Ben.

[Ben One]

Glad the links are useful! And that's right, I do have a physics background (just a B.S.--glad there are other physicists here too!)

 

Yes, one area of mystery is exactly HOW the higher levels of the brain process sound and help us separate multiple sound sources into individually perceivable sounds--right now it does seem like a binaural matrix . Much more is known about how the inner ear works at this point.

 

This is somewhat OT, but I thought this was another fun topic:

 

Did you ever wonder how even on a cheap AM radio, a song sounds musically pretty much the same as it does on a high end stereo system, yet the tinny radio is not able to play the lowest and most important notes of the song? Humans somehow have the ability to "fill in" those low notes based on the higher notes that they hear on the song.

 

A somewhat recent paper says that we don't need the brain to do this, that all the required machinery is in our ear:

 

http://www.nature.com/news/1999/990708/pf/990708-7_pf.html

[Bruce Swedien]

WOW!!!!!

 

I'm impressed! You folks are now all admitted to the Royal Order of Top Drawer Gassers!!!

 

I think that this is the real deal---->Ken/Eleven Shadows

 

It's all about making choices, often subjective, artistic, creative choices.

 

Brucie sez---->I would go so far as to say - ALMOST always personally biased, or better yet - ALMOST always PERSONAL...

 

miki sez---->Given the information above, doesn't it then stand to reason that each person's perception is different therefore, how one engineer/technician perceives/interprets the sound may be significantly different than how another perceives it? Isn't the 'real trick' then more of how each individual can better perceive the sounds rendered in order to produce better recordings and wouldn't that be extremely subjective rather than objective science?

 

Brucie sez---->Exactly - BUT what difference does that make? In my estimation - NOT A BIT!!!

 

I find it very interesting to look back at the evolution of automobile and home sound equipment from the 1950's up to today. If we carefully examine the curve of change and improvement in automobile and home sound for that period of time, we can see, as a parallel, a definite improvement in the listening discernment of the general public. If we look at, and listen to the car radios and the home sound systems of the early 1950's we will see that the equipment looks simple and primative, we will hear that a radio of that time only picks up AM signals and has merely very primative volume and tone controls.

 

Has the perception and preference of the public improved, when it comes to the sound of music in the car and at home? Or is it a conscious effort on the part of the public and the equipment manufacturers towards "Reality", hence the term High-Fidelity".

 

Does reality in sound, or "High-Fidelity" mean that we wish to listen to an unaltered acoustical event. I don't think such is the case at all. I think the importance of high quality sound is a cultural phenomonen, not a physiological requirement.

 

I think that there are several aspects to the revolution, or phenomonen in the music lover's listening ability and the resultant upgrading of music reproduction equipment. Such facets as; "Taste", "Style", "Fashion", "Asthetic Sensibility", and "Culture".

 

I don't think intelligibility of message is a factor at all, because the quality of intelligibility is present from a cheap car radio to a state-of-the-art sound system. If we were to say that sound quality preferences in the listener are based on realism, then we assume to take on the impossible task of defining realism in sound. In todays music then, we would have to add to that the additional problem of electronic fabrication or manipulation of sounds and music.

 

I brought this little subject to your attention to make you think about how important "Critical Listening' is to what we do in the studio.

 

By "Critical Listening" I mean that - We must first know what music sounds like in order to make a meaningful recording for the public to enjoy, or better yet to dash out to a good record store(If there are any left!) and plunk down hard earned cash and purchase that G____ D___ record!

 

I am appalled at how many so-called professionals in our industry try to learn their craft by listening to records! None of these people have a clue as to what music really sounds like!

 

Common meanings, values and experiences, to share with others, create a learning experience that is undeniable. We learn to hear clearly with our ears, and then accurately evaluate what we are hearing with our experiences.

 

The importance, of reproduced sound, lies not in any inherent acoustical or technical value, but in what it signifies to the soul of the listener.

 

Bruce Swedien

[Kendrix]

Originally posted by Bruce Swedien:

WOW!!!!!

 

I'm impressed! You folks are now all admitted to the Royal Order of Top Drawer Gassers!!!

 

This has made my day.

Its truly an honor and a privledge - I think.

 

But wait...Who else is in the Royal Order of Top Gassers?

I think it might be an interesting list.

 

Do we get any discounts on tube traps?

When are the meetings held?

There arent any dues to pay are there?

Will you send us all certificates of membership?

[Chip McDonald]

Originally posted by Bruce Swedien:

Think of this for a minute....

 

Ihave you ever noticed how, when you monophonically record a person speaking in a reverberant room, (Like on a small mono cassette recorder) the recording sounds highly reverberant and essentially unintelligible. Yet when you are in that very same room with that person speaking, you can understand every word.

Having two information sources allows for a reduction of differences between the two sources, leaving what remains as the constant between the two sources.

 

I think that in a perfectly symetrical room with the sound source positioned perfectly centered relative to the surrounding reflective surfaces, you still get a blurring of separation between originating source and secondary reflections... as opposed to reality (outside of a basic digital reverb) where one side is almost always going to have some sort of reverberant information that is unique relative to the other.

 

So, the useful application of this is to think in terms of using like reverbs in stereo when wanting to deemphasize the sound source, and dissimiliar when wanting to localize the sound source.

 

Which makes me think that an interesting effect would be to create a reverb that alters it's stereo-uniqueness depending on input level. Which, if you think about it, is again reality; you tend to want to hear a louder and louder sound when evaluating a room, since a low-level sound tends to blend into the reverberant field more.

 

As such, digital reverbs that don't alter their stereo-symmetry depending on level are not going to be perceived as realistic.

 

So now I'm thinking, one should have a pair of somewhat-similar stereo reverbs, split with an expander so that when the levels cross a threshold a reverb with an eq smearing things around on one side relative to the other, with less symmetry in early reflections (*not* more vs. less, but *asymetrical* versus more symetrical and spectrally similar) and so forth, would be an interesting thing to try.

 

Bleh, sorry, just thinking outloud.... back to my cave...

[Jimbroni]

So now I'm thinking, one should have a pair of somewhat-similar stereo reverbs, split with an expander so that when the levels cross a threshold a reverb with an eq smearing things around on one side relative to the other, with less symmetry in early reflections (*not* more vs. less, but *asymetrical* versus more symetrical and spectrally similar) and so forth, would be an interesting thing to try

Excellent, a simplified version of what I was trying to do, but instead of using measurements and equations to develop a new way to model acoustic effects, you're using standard gear to deliver a similar result by ear.