Understanding the mechanisms of multimodal communication in deaf children with cochlear implants

Children develop and interact in complex, multimodal environments.

I will investigate how deaf children with cochlear implant (CIs) integrate visual information from the talker’s face to supplement sound.

This is important because CIs provide a degraded sound that lacks pitch and much affective information, placing high cognitive demand on the developing brain and increasing reliance on visual information and audio-visual integration. I will conduct experiments comparing adolescents with early-onset deafness and bilateral CIs to hearing controls.

Eye tracking will be used to understand allocation of visual attention during word and emotion perception and the interaction between these processes.

Visual evoked potentials will be measured using electroencephalography to understand the efficiency of cortical processing during visual perception.

Psychophysics and pupillometry will be combined to estimate the benefit gained from audio-visual speech integration and the impact on cognitive effort.

Functional near-infrared spectroscopy will enable cortical imaging of CI users without interference from CI-generated artefacts.

Correlations between activity in temporal and occipital cortices will be examined to understand audio-visual cortical connectivity and to specify mechanisms of audio-visual integration.

This work will enable us to better understand the role for visual cues in augmenting current auditory-only rehabilitative interventions to help deaf children communicate effectively.