Sensory Neuroengineering

Tobias Reichenbach
Department of Bioengineering, Imperial College London

Recent work

Enhancement of speech-in-noise comprehension through neurostimulation

neurostimulation and speech

Transcranial current stimulation can influence neuronal activity. As a striking example, current stimulation paired to a speech signal, with the current following the speech envelope, can influence the comprehension of speech in background noise. The speech envelope is thereby a slowly-varing signal with frequency contributions that lie mostly in the delta and in the theta frequency bands. Here we show that the modulation of speech comprehension results from the theta band, but not from the delta band. Moreover, we find that the theta-band stimulation without an additional phase shift improves the speech comprehension as compared to a sham stimulus.

M. Keshavarzi, M. Kegler, S. Kadir, T. Reichenbach,
Transcranial alternating current stimulation in the theta band but not in the delta band modulates the comprehension of naturalistic speech in noise,
Neuroimage (2020) 210:116557. [pdf]

Measuring speech comprehension from EEG recordings

comprehension decoding

If hearing aids could measure how well a wearer understands speech, they might be able to optimize and adapt their signal processing to enable the best user experience. Hearing aids can potentially measure brain responses to speech from electrodes, but how this can inform on speech comprehension has remained unclear. Here we report significant progress on this issue. By combining machine learing with an experimental paradigm that allows to disentable lower-level acoustic brain responses from neural correlates of the higher-level speech comprehension, we show that speech comprehension can be decoded from scalp recordings.

O. Etard and T. Reichenbach,
Neural speech tracking in the theta and in the delta frequency band differentially encode clarity and comprehension of speech in noise,
J. Neurosci. 39:5750 (2019). [pdf]

Decoding attention to speech from the brainstem response to speech

ear eeg

We are often faced with high noise levels: in a busy pub or restaurant, for instance, many conversations occur simultaneously. People with hearing impairment typically find it difficult to follow a particular conversation, even when they use hearing aids. Current aids do indeed amplify all the surrounding sounds, not only the target. If a hearing aid could know which speaker a user aims to listen to, it could amplify that voice in particular and reduce background noise. Here we show that a hearing aid can potentially gain knowedlege of a user's attentional focus from measuring the auditory brainstem response from surface electrodes. We show in particular that short recordings, down to a few seconds, and a few scalp electrodes suffice for a meaningful decoding of auditory attention.

O. Etard, M. Kegler, C. Braiman, A. E. Forte, T. Reichenbach,
Real-time decoding of selective attention from the human auditory brainstem response to continuous speech,
Neuroimage 200:1 (2019). [pdf] [bioRxiv]

Neural responses to speech can help to diagnose brain injury

Brain imaging

Brain injury such as following traffic or sports accidents can lead to severe disorders, including disorders of consciousness. This disorder is currently diagnosed through behavioural assessments, but this method fails when patients are not able to respond overtly. We investigated whether neural responses to speech as measured from the clinically-applicable EEG can aid to diagnose disorders of consciousness. We focussed on the neural tracking of the speech envelope that can index attention to speech as well as speech comprehension. We find that the latency of the neural envelope tracking related to the severity of the disorder of consciousness: patients in a vegetative state without signs of consciuosness showed neural responses to the speech envelope that were significantly delayed compared to patients that exhibited consciusness.

C. Braiman, E. A. Fridman, M. M. Conte, C. S. Reichenbach, T. Reichenbach, N. D. Schiff
Cortical Response to the Natural Speech Envelope Correlates with Neuroimaging Evidence of Cognition in Severe Brain Injury,
Curr. Biol. 28:1-7 (2018). [pdf]

How we can tune in to a voice in background noise

The investigators

In order to focus on a particular conversation, listeners need to be able to focus on the voice of the speaker they wish to listen to. This process is called selective attention and has been extensively studied within the auditory cortex. However, due to neural feedback from the cortex to lower auditory areas, the auditory brainstem as well as the inner ear, these structures may already actively participate in attending to a particular voice.

We have devised a mathematical method to measure the response of the auditory brainstem to the pitch of natural speech. In a controlled experiment on selective attention, we have then shown that the brainstem responds stronger to the pitch of the voice that a person is listening to than to that of the ignored voice. Our findings demonstrate that the brainstem contributes already actively to selective attention. They also show that the pitch of a voice can be a powerful cue to focus on that voice, which may inspire future speech-recognition technology.

A. E. Forte, O. Etard and T. Reichenbach,
The human auditory brainstem response to running speech reveals a subcortical mechanism for selective attention,

eLife 6:e27203 (2017). [pdf] [bioRxiv]

Shape of hair bundles of inner and outer hair cells

Hair bundles

The mammalian sense of hearing relies on two types of sensory cells: inner hair cells transmit the auditory stimulus to the brain, while outer hair cells mechanically modulate the stimulus through active feedback. Stimulation of a hair cell results from displacement of its mechanosensitive hair bundle. While hair bundles of inner hair cells are of linear shape, those of outer hair cells exhibit a distinctive V-shape. The biophysical rationale behind this morphology, however, has remained unknown.

We have used analytical and computational methods to study the fluid flow across rows of differently shaped hair bundles. We found that rows of V-shaped hair bundles have a considerably reduced resistance to crossflow, and that the biologically observed shapes of hair bundles of outer hair cells are near-optimal in this regard. This observation accords with the function of outer hair cells. Our work thereby suggest a biophysical rationale of the different shapes of hair bundles in the mammalian ear.

N. Ciganovic, A. Wolde-Kidan, T. Reichenbach,
Hair bundles of cochlear outer hair cells are shaped to minimize their fluid-dynamic resistance,

Sci. Rep. 7:3609 (2017). [pdf]

Mechanics of the inner ear at low frequencies

Organ of Corti

Low frequency sounds, below four kilohertz, are used by human ears for perceiving speech. However, the exact mechanism for how the inner ear processes these sounds is poorly understood, as the organ is difficult to access in experiments. Recently we have proposed that low-frequency hearing may employ unidirectional amplification: the mechanosensitive hair cells then amplify vibration but do not feed back to the inner ear's basilar membrane (Reichenbach and Hudspeth, PNAS 2010).

We have now employed laser-interferometric measurements in vitro as well as optical-coherence tomography in vivo to investigate cochlar mechanics at low frequencies. We find that the basilar membrane moves only little, and the motion is confined to a small region below the outer hair cells. These findings suggest that mechanical amplification by outer hair cells does not feed back significantly to the basilar membrane as hypothesized earlier. Our experiments open a route for a comprehensive understanding of low-frequency cochlear mechanics.

R. L. Warren, S. Ramamoorthy, N. Ciganovic, Y. Zhang, T. Wilson, T. Reichenbach, A. L. Nuttall, A. Fridberger, Minimal basilar membrane motion in low-frequency hearing, Proc. Natl. Acad. Sci. U.S.A. 113:4303 (2016). [pdf]

Image by by R. L. Warren.

Auditory brainstem response to continuous monotone speech

Monotone speech

Speech has voiced part, the spectrogram of which are characterized by a fundamental frequency and higher harmonics. The fundamental frequency lies typically between 100 and 250 Hz for men, and can be somehwat higher for women. Previous experiments on many repated presentations of the same voiced element of speech have shown that neurons in the auditory brainstem respond at the fundamental frequency, but it has remained unclear what role this brainstem response plays in speech processing.

We habe begun to investigate this issue by recording brainstem responses to continuous, non-repetitive speech signals. To ease the read-out of the brainstem response, we have made speech monotone. The fundamental frequency that normally varies is thus constant. We show that the brainstem responds at that constant frequency, and that this response is larger for unintelligible speech played in reverse than for the standard intelligible speech played in a forward manner. Further experiments are required to determine whether this difference signals a modulation of the brainstem response by speech intelligibility or differences in the acoustic structure between forward and reverse speech.

C. Reichenbach, C. Braiman, N. Schiff, A. J. Hudspeth, and T. Reichenbach, The response of the auditory brainstem to the spectral structure of speech reflects language comprehension and attention, Front. Comp. Neurosci. 10:47., Front. Comp. Neurosci. 10:47 (2016). [pdf]

Neural response to the beat of music

EEG setup

Cortical oscillations in the delta and theta frequency band are known to track the envelope of an attended speech signal. Music has similar spatio-temporal characteristics as speech. Do cortical oscillation accordingly play a role in music perception as well?

We have designed an experiment to begin to explore this hypothesis. Using well-controlled musical stimuli we show that the neural acticity of the cerebral cortex responds to the beat of music. We further show that this steady-state brain response informs on the comprehension of music as well as on attention. Our findings can lead to a better diagnosis of amusia, a disorder in the processing of music, as well as to a better assessment of cognition and awareness in patients with traumatic brain injury.

B. Meltzer, C. S. Reichenbach, C. Braiman, N. D. Schiff, A. J. Hudspeth, T. Reichenbach, The steady-state response of the cerebral cortex to the beat of music reflects both the comprehension of music and attention, Front. Hum. Neurosci. 9:436 (2015). [pdf]

Review on the biophysics of hearing

hair bundle We have written a comprehensive review on the active biophysics of hearing that has been published in Reports on Progress in Physics. The review gives an accessible presentation of the intricate functioning of the inner ear, from its nonlinear fluid dynamics to active amplifcation in hair cells and their mechanosensitive ion channels. We hope that the review serves new students and researchers entering the field. Established researchers may benefit from an exposition of current controversies and open topics.

T. Reichenbach A. J. Hudspeth, The physics of hearing: fluid mechanics and the active process of the inner ear, Rep. Progr. Phys. 77:7 (2014). [pdf]

The review has been recommended and featured by the Faculty of 1000.

Bone conduction and otoacoustic emissions

bone conduction Many of us may not be aware that our skull also transmits sound to our ears. This is why we cringe when we hear a recording of our own voice: normally we hear our voice both through air as well as through bone conduction. When we listen to a recording, the bone conduction is missing, which makes our voice sound different. However, the mechanism of bone conduction remain poorly understood. This contrast with recent commercial usage of bone conduction in headphones, notably in the new Google Glass device.

In a new mathematical and computational study in Nat. Commun. we show that sound can deform the bone that houses the inner ear. These deformations can travel as waves and elicit a hearing sensation. We show that the same deformation of the bone can also play a role in the emission of sound form the ear, so-called otoacoustic emission. These are important clinically as they can be used to assess healthy hearing.

The work is covered in a popular-science radio broadcast on Rundfunk Berlin-Brandenburg (rbb, Germany). The podcast, with Dr. T. Tchumatchenko, is available here (in German). The work has also been featured in a news story by Imperial College as well as in a news story by the Max-Planck Society.

T. Tchumatchenko, T. Reichenbach, A cochlear-bone wave can yield hearing sensation as well as otoacoustic emission, Nat. Commun. 5:4160 (2014). [pdf]