The term cocktail party effect has been around for several decades. However, only during the last several years have researchers gained an in-depth understanding of the phenomenon. Also known as selective hearing, the cocktail party effect refers to the ability of humans to focus all their attention on one speaker while tuning out competing and distracting noises in the background.
Human wiring appears to make focusing attention on the person they want to hear almost an automatic process. This is true regardless of the environment, whether it is a noisy bar, sporting event, work, school, or a literal cocktail party.
Thanks to rapidly advancing technology, researchers better understand how the phenomenon of selective hearing works in the brain. As a result, they have greater knowledge of brain functions and the causes of the cocktail party effect phenomenon.
The 2012 UCSF Study on the Cocktail Party Effect
Nine years ago, two researchers at the University of California at San Francisco worked with three patients who had severe epilepsy. They sought to uncover the cause of electrical signal mix-ups in the brain causing severe seizures. These episodes were important to dissect because of their negative impact on quality of life.
Dr. Edward Chang, a UCSF faculty member at the Keck Center for Integrative Neuroscience and Department of Neurological Surgery, led the study. Post-doctoral fellow Nima Mesgarani worked with Dr. Chang throughout this research study.
Prior to working with Dr. Change and Nima Mesgarani, PhD, the three people with epilepsy underwent brain mapping over a seven-day period with the Epilepsy Team from UCSF. The team obtained the brain mapping report by attaching up to 256 tiny electrodes under each patient’s skull on the outer surface or cortex of their brain. The electrodes recorded activity from the auditory cortex in the temporal lobe.
Back in 2012, Dr. Chang commented that the availability of brain recordings with high resolution and algorithms containing powerful decoding capabilities created new and exciting research possibilities with the human brain. Technology has only advanced since that time, which means manufacturers of sound amplification and voice-activated devices will develop increasingly effective conversation boosting technology.
How the Brains of Research Subjects Responded to Speech
Dr. Chang and post-doctoral fellow Nima Mesgarani played two speech samples for each of the patients and asked them to pay special attention to specific phrases. The researchers then asked the patients to repeat the key phrases spoken by just one of the speakers.
Using newly identified brain decoding methods, Chang and Mesgarani reconstructed what the three patients heard just by analyzing activity patterns in their brains. They were surprised to learn that the neural responses in the auditory cortex of all three patients detected only the key phrases from the selected speaker.
Cocktail-Party Effect: The ability to attend to one of several speech streams while ignoring others, as when one is at a cocktail party. Researchers first thought unattended messages aren’t processed, but later findings indicated that meaning is identified in at least some cases. pic.twitter.com/Dury6xdZpM
— AP Psychology Review (@Psych_Review) November 2, 2020
The decoding algorithm used by the researchers could predict with amazing accuracy which speaker and which words the subjects paid attention to the most. This was due to the algorithm’s interaction with neural patterns in the brain that indicated when the attention of the epilepsy patients started to turn elsewhere. The researchers were also surprised to see that the algorithm was so accurate that it knew when the patients turned their attention to the wrong words.
The 2016 Research Study – Furthering What We Know About the Cocktail Party Effect
Four years after the UCSF study, Hodgraf et al worked with a different group of epilepsy patients who also had electrodes attached to the surface of their brain to track the timing and severity of seizures. The researchers in this study learned that the human brain could isolate speech from garbled and faulty input. This proved their theory that the brain automatically emphasizes speech sounds over other types of noise.
The Hodgraf et al research team first presented the patients with a garbled message. They received a clear and filtered message next followed by another message that would be difficult for anyone to understand. The study participants could understand all three messages while displaying nearly the same language-appropriate neural activity in the brain. They also displayed auditory tuning and feature detection in their spectrotemporal region.
The Impact of Speech Recognition on the Human Brain and Artificial Intelligence
Both research studies indicated that speech sounds reaching the auditory cortex reflect more than external acoustic environments. In conclusion, the sounds of other people’s speech are an accurate reflection of what humans prefer to hear.
The studies also made it possible for researchers to gain a better understanding of how the brain processes language. With this understanding, they have greater insight into the study of brain-based disorders as well as the cocktail party effect. Examples include:
- Alzheimer’s disease
- Attention deficit disorder
- Autism
- Dementia
- Language learning disorders
Dr. Chang expressed optimism following the USCF study that researchers would eventually create neuroprosthetic devices using this technology.. The device would decode thoughts for people with a wide range of cognitive disorders or without the ability to speak.
In-depth knowledge of brain wiring explains why people naturally favor some types of auditory cues over others. This knowledge inspires novel approaches toward the automation and improvement of how voice-activated devices filter sounds. Similarly, it helps detect when people are issuing verbal commands.
Manufacturers of voice-activated technology have a vested interest in the research findings about the human brain and auditory processing. Nonetheless, the human hearing frequency range and processing are far more sophisticated than AI-powered speech recognition devices.
Mesgarani, the engineer and post-doctoral fellow who worked with Dr. Chang, has a background in the research of automatic speech recognition. He concedes that humans inherently possess natural speech recognition capabilities. They can therefore focus on a single voice against a backdrop of multiple competing sounds better than voice recognition devices.
However, machine emulation of the cocktail party effect phenomenon grows stronger every year. This is good news for the millions of people around the world who suffer from hidden hearing loss and could benefit from affordable hearing devices that filter out unwanted sounds.
