István Winkler, scientific advisor at the Institute of Cognitive Neuroscience and Psychology at the HUN-REN Research Centre for Natural Sciences (HUN-REN RCNS) and Susan L. Denham, researcher at the Bournemouth University, created a new framework linking the mechanisms of sound source separation with information processing in everyday life, such as forming and recalling memories of real-life scenes. This novel theory opens the way for developing qualitatively improved computer models of auditory perception and episodic memory. The theory was published in the distinguished journal Nature Reviews Psychology.
Everyone has experienced talking with someone on the street (see illustration), while cars, other people, etc., passed by, with perhaps a street musician playing in the background. Sounds from all these sources are mixed together before arriving at our ears. Thus, separation of the concurrently active sound sources, for example, to select and track the voice of our partner, is one of the most important and most difficult tasks of the auditory system. There is yet no consensus amongst researchers about what processing steps our brain employs to reach these goals.
The new theory helps to solve this problem. It describes the information carried by sounds that allow the brain to distinguish and, if possible, identify the actual sound sources and their sound-generating actions. (On the illustration: speech, car noises, music; our friend speaks, a car breaks, music emerges from the shop.) The paper discusses the operations and algorithms possibly leading to these difficult – though subjectively easy – separations and identifications.
One important feature of the theory is that it fits auditory information processing into the Event Segmentation Framework, which describes the processing of everyday experiences. This fills an important hole within the field of perceptual/cognitive psychology. Linking the two theories allows assessing how auditory perception feeds into the processing of everyday scenes and events and the formation of memories of them. Further, it helps to explain the effects of information from other modalities (e.g., vision, touch) and long-term memory on auditory perception.
Linking perception and cognition promotes connecting computer models separately developed within the two fields. This new generation of models can link auditory perception and general cognition by providing the models of the latter, which have mostly been based on symbolic information, with actual sensory information. These computer models could help individuals with hearing problems (such as most elderly people), find applications in telecommunication (such as joining conversations taking place in real space through computer networks), and improve man-machine communication in noisy environments.
Illustration: People walking and talking on a busy street