Multisensory Processing of Human Speech Measured with msec and mm Resolution

以毫秒和毫米分辨率测量的人类语音的多感官处理

• 批准号: 8821464
• 负责人: DANIEL YOSHOR
• 金额: --
• 依托单位: MICHAEL E DEBAKEY VA MEDICAL CENTER
• 依托单位国家: 美国
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-04-01 至 2018-03-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8821464
• 关键词: Acquired Deafness; Advisory Committees; Affect; Area; Attention; Auditory; Auditory area; Bayesian Modeling; Biology; Brain; Cerebral cortex; Communication; Coupling; Electrocorticogram; Epilepsy; Etiology; Face; Frequencies; Functional Magnetic Resonance Imaging; Functional disorder; Goals; Hearing; Human; Impairment; Implanted Electrodes; Knowledge; Language; Lead; Measures; Methods; Modality; Modeling; Neurons; Noise; Patients; Perception; Phase; Process; Relative (related person); Research; Resolution; Sensorineural Hearing Loss; Sensory; Services; Signal Transduction; Speech; Speech Perception; Stimulus; Stroke; Structure of superior temporal sulcus; Techniques; Testing; Therapeutic Intervention; Tinnitus; Translating; Traumatic Brain Injury; Veterans; Visual; Visual Cortex; Voice; abstracting; behavior measurement; central nervous system injury; cognitive ability; cognitive function; directed attention; disability; experience; extrastriate; extrastriate visual cortex; flexibility; hearing impairment; hemodynamics; human subject; improved; innovation; millimeter; millisecond; multisensory; neuromechanism; public health relevance; relating to nervous system; response; temporal measurement; therapy design; visual information; visual stimulus

DESCRIPTION (provided by applicant): Project Summary/Abstract Speech perception is one of the most important cognitive functions of the brain. Humans use both the auditory information available in the heard speech and the visual information available from viewing the speaker's face to understand speech. Functional magnetic resonance imaging (fMRI) is the most popular method for examining human brain function, but suffers from the critical limitation of poor temporal resolution: because it measures hemodynamic changes, it has only ~sec resolution, rather than the ~ms timescale of neuronal activity. This renders fMRI a poor choice for examining dynamic neuronal processing. Instead, we propose to use electrocorticography (eCog), in which electrodes implanted in human subjects for the treatment of epilepsy are used to directly measure neuronal activity in the human brain. The major innovation of this proposal will be to use eCog to measure the dynamic neural processes underlying audiovisual speech perception. The central hypothesis is that rapid communication occurs between the human superior temporal sulcus (STS) and sensory areas through neural oscillations, specifically frequency-specific interactions between the multisensory cortex in the STS and earlier sensory areas in the auditory and visual cortex. As a measure of functional connectivity between areas in the speech network, we will measure trial-by-trial correlations in gamma-band power between brain areas. We assume that high correlations in gamma- band power reflect information transfer and a functional connection between the areas. Neuronal oscillations, particularly in the gamma range (~ 30 - 200 Hz), have been found to reflect neuronal spiking activity and several studies suggest that neuronal oscillations might be an important mechanism of information transfer in the brain that modulates spiking activity. We will test the hypothesis that gamma-band activity in the STS correlates with gamma-band activity in auditory association areas during multisensory speech perception. In ordinary speech, the auditory speech signal is much more informative than the visual speech signal. Therefore, we hypothesize that for clear audiovisual speech, the correlation in gamma oscillations between auditory cortex and STS is stronger than the correlation in gamma oscillations between visual cortex and STS. Conversely, we expect a stronger correlation between gamma-band activity in the visual cortex and the STS when the auditory speech signal is noisy. The proposed research will make use of direct neural recordings from the human brain to investigate whether the information content of the different speech signals affects the communication between early sensory areas and multisensory associative areas. These findings will help us understand the neuronal dynamics involved in speech perception and devise new therapies to help treat veterans with hearing loss and other disabilities.

描述（由申请人提供）： 项目摘要/摘要言语感知是大脑最重要的认知功能之一。人类利用听到的语音中提供的听觉信息和通过观察说话者的面部获得的视觉信息来理解语音。功能磁共振成像 (fMRI) 是检查人脑功能最流行的方法，但受到时间分辨率差的严重限制：因为它测量 血流动力学变化，它只有~秒的分辨率，而不是~ms的神经元活动时间尺度。这使得功能磁共振成像成为检查动态神经元处理的糟糕选择。相反，我们建议使用皮层电图描记术（eCog），其中植入人体用于治疗癫痫的电极直接测量人脑中的神经元活动。该提案的主要创新是使用 eCog 来测量视听语音感知的动态神经过程。核心假设是，人类颞上沟 (STS) 和感觉区域之间通过神经振荡进行快速通信，特别是 STS 中的多感觉皮层与听觉和视觉皮层中的早期感觉区域之间的频率特定相互作用。作为语音网络中区域之间功能连接的衡量标准，我们将测量大脑区域之间伽玛波段功率的逐次试验相关性。我们假设伽马能带功率的高相关性反映了信息传输和区域之间的功能连接。神经元振荡，特别是在伽马范围（~ 30 - 200 Hz）内，已被发现反映神经元尖峰活动，并且一些研究表明神经元振荡可能是大脑中调节尖峰活动的信息传递的重要机制。我们将检验这样的假设：在多感官语音感知过程中，STS 中的伽马带活动与听觉关联区域的伽马带活动相关。在普通语音中，听觉语音信号比视觉语音信号包含更多信息。因此，我们假设对于清晰的视听语音，听觉皮层和STS之间的伽马振荡的相关性强于视觉皮层和STS之间的伽马振荡的相关性。相反，当听觉语音信号有噪声时，我们预计视觉皮层中的伽马带活动与 STS 之间的相关性更强。拟议的研究将利用人脑的直接神经记录来调查不同语音信号的信息内容是否影响早期感觉区域和多感觉联想区域之间的交流。这些发现将帮助我们了解与言语感知有关的神经元动力学，并设计新的疗法来帮助治疗患有听力损失和其他残疾的退伍军人。