Dynamic Neural Mechanisms of Audiovisual Speech Perception

视听言语感知的动态神经机制

• 批准号: 9356348
• 负责人: CHARLES E SCHROEDER
• 金额: $ 94.55万
• 依托单位: COLUMBIA UNIVERSITY HEALTH SCIENCES
• 依托单位国家: 美国
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-30 至 2019-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9356348
• 关键词: Academic Medical Centers; Auditory; Auditory area; Brain; Cognitive; Complex; Computer Simulation; Cueing for speech; Cues; Data; Dyslexia; Electric Stimulation; Electrocorticogram; Electroencephalography; Elements; Engineering; Epilepsy; Ethicists; Face; Foundations; Frequencies; Functional Magnetic Resonance Imaging; Gestures; Goals; Hand; Hearing; Homing; Human; Income; Individual; Judgment; Learning Disabilities; Long Island; Medical center; Medicine; Methods; Modality; Modeling; Movement; Neurologist; Neurons; Neurosurgeon; Operative Surgical Procedures; Oral cavity; Patients; Pattern; Perception; Phase; Physiological Processes; Play; Population; Posture; Process; Research Personnel; Schizophrenia; Science; Sensory; Site; Speech; Speech Perception; Stimulus; Stream; Stroke; Structure of superior temporal sulcus; Testing; Time; Visual; Visual Cortex; Voice; Work; audiovisual speech; autism spectrum disorder; base; behavior influence; college; follower of religion Jewish; improved; instrument; language processing; millisecond; multisensory; nervous system disorder; neuromechanism; relating to nervous system; skills; social; speech processing; temporal measurement; transmission process; visual information; visual speech

ABSTRACT – (Title: Dynamic Neural Mechanisms of Audiovisual Speech Perception) Natural speech perception is multisensory; when conversing with someone that we can see, our brains combine visual (V) information from face, postural and hand gestures with auditory (A) information from the voice. The underlying speech processing is extremely rapid, with incoming AV units (e.g., syllables) arriving every few hundred milliseconds that must be encoded and passed on before the next syllable arrives. Finally, this bottom-up sensory information is combined with a top-down cognitive component: what we perceive is strongly influenced by its context. Speech is fundamentally human, and thus, its brain mechanisms are usually studied with noninvasive fMRI, EEG and MEG. Because each method has critical limitations in spatial or temporal resolution, identifying the specific brain mechanisms of speech perception - AV integration, precise and rapid information encoding and top-down control - is a nearly intractable problem. This three-year U01 project will sidestep the problem using direct recording of neuron ensemble (electrocorticographic or ECoG) activity and single neuron activity, along with direct stimulation of selected sites in the brains of surgical epilepsy patients as they process AV speech. Our collaborative ECoG team embodies expertise in multisensory integration and speech perception and leverages the skills and perspectives of neuroscientists, neurosurgeons, engineers, neuropsychologists, neurologists, and ethicists across three leading epilepsy centers: Columbia University Medical Center, Baylor College of Medicine and Northshore-Long Island Jewish Medical Center. By combining the expertise and patients available at all three centers, we will be able to tackle problems that are inaccessible to individual investigators. Our overarching hypothesis, building on our past work and supported by preliminary data, is that fluctuations in the excitability of neurons—oscillations—play a key role in speech processing. Aim 1 tests the general hypothesis that delta/theta range (2-8 Hz) neuronal oscillations play a key role in the integration of auditory and visual speech information. Aim 2 tests the general hypothesis that high-frequency activity (50 Hz and above) encodes representations of auditory and visual speech information, reflecting both bottom-up and top-down influences on perception. The concept employed in this proposal of oscillatory dynamics as mechanistic instruments (Aim 1) that organize the encoding of information in neuronal firing patterns under dynamic top-down control (Aim 2) are part of a paradigm shift in speech science. The broad goal of this proposal is to contribute key foundations for this new paradigm, and set the stage for a comprehensive understanding of the brain circuits and physiological processes underlying natural speech perception, including complex social settings.

摘要 - （标题：视听语音感知的动态神经机制） 自然语音感知是多感的。当与我们看到的人交谈时，我们的大脑 将视觉（v）的信息结合在一起，来自面部，姿势和手势的信息与听觉（a）信息（a）信息 嗓音。潜在的语音处理非常迅速，传入的AV单元（例如，音节）到达 在下一个音节到达之前，必须编码并传递几百毫秒。最后， 这种自下而上的感官信息与自上而下的认知组成部分结合在一起：我们的感知是 受其背景的强烈影响。语音从根本上是人类，因此，其大脑机制通常是 研究了非侵入性fMRI，脑电图和梅格。因为每种方法在空间或 临时分辨率，确定语音感知的特定大脑机制 - AV整合，精度 快速的信息编码和自上而下的控制 - 几乎是一个棘手的问题。 这个为期三年的U01项目将使用神经元合奏的直接记录回避问题 （皮质学或ECOG）活性和单个神经元活性，并直接刺激选定 手术癫痫患者的大脑中的站点在处理AV语音时。我们的协作ECOG团队 体现多感官整合和语音感知方面的专业知识，并利用技能和 神经科学家，神经外科医生，工程师，神经心理学家，神经科医生和伦理学家的观点 在三个领先的癫痫中心：哥伦比亚大学医学中心，贝勒医学院和 北岸长岛犹太医疗中心。通过将所有三个可用的专业知识和患者相结合 中心，我们将能够解决个人调查人员无法访问的问题。 我们的总体假设是基于我们过去的工作并得到初步数据支持的，这是 神经元的兴奋（振荡）中的波动在语音处理中扮演关键作用。 AIM 1检验了一个总体假设，即Delta/Theta范围（2-8 Hz）神经元振荡起着关键作用 听觉和视觉语音信息的集成。 AIM 2检验了一般假设，即高频活动（50 Hz及以上）编码的表示 听觉和视觉语音信息，反映了自下而上的和自上而下的影响。 该概念是在振荡动力学的这一提议中作为机械工具的（目标1） 在动态自上而下控制下（AIM 2）以神经元射击模式组织信息的编码（AIM 2） 言语科学范式转变的一部分。该提案的广泛目标是为关键基础贡献 对于这个新的范式，并为全面了解大脑电路和 自然语音感知的基础的物理过程，包括复杂的社会环境。