CRCNS Research Proposal: Novel computational approaches for neural speech prostheses and causal dynamics of language processing

CRCNS 研究提案：神经语音假体和语言处理因果动力学的新型计算方法

• 批准号: 2309057
• 负责人: Yao Wang
• 金额: $ 95万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2309057&HistoricalAwards=false
• 关键词: CRCNS; Research; Proposal; Novel; computational

This project aims to develop computational approaches that will allow for a deeper understanding of the neurobiology of language and translate to novel clinical applications for speech and language. The computational research is necessary to enable future speech neuroprostheses, which would allow patients with degenerative conditions or neurological damage to drive a speech synthesizer using their intact cortical structures. Additionally, the combination of tools for analyzing connectivity and regions critical for language in individual brains will provide insight into the network dynamics of language cortex and open the door to replacing the clinical practice of using electrical stimulation to map language-critical regions, which is not well tolerated by all patients.The research is framed across three intertwining thrusts. The first thrust will explore the use of deep learning for decoding produced speech from various neural signals captured by intracranial depth and surface electrodes. This thrust will develop models within and across multiple patients that robustly decode speech while overcoming current limitations in the field, leading to potential integration into speech neuroprostheses. The second thrust will explore efficient algorithms for estimating brain connectivity dynamics from the recorded signals. This thrust will develop novel techniques that estimate directed connectivity among a large number of recording sites, which is essential for understanding the dynamic interactions across cortex during cognitive processing. The third thrust will develop deep-learning models that can predict brain regions that are causally critical for language processing based on the neural recordings alone. These models will serve to pinpoint brain regions that are critical for language processing without needing to electrically stimulate the brain. Taken together, the proposed research leverages recent innovations in deep learning (e.g. transformers, graph neural networks, self-supervised learning) to overcome challenges stemming from the non-structured and varied placements of electrodes across patients, data scarcity, scalability and stability. These new approaches will be shared with the scientific community as open-source and replicable technologies. This project is jointly funded by the Collaborative Research in Computational Neuroscience (CRCNS) program and the Disabilities and Rehabilitation Engineering Program (DARE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在开发计算方法，使人们能够更深入地了解语言的神经生物学，并将其转化为语音和语言的新型临床应用。计算研究对于实现未来的语音神经假体是必要的，这将使患有退行性疾病或神经损伤的患者能够利用其完整的皮质结构驱动语音合成器。此外，用于分析个体大脑中的连通性和语言关键区域的工具组合将提供对语言皮层网络动态的深入了解，并为取代使用电刺激来映射语言关键区域的临床实践打开了大门，这不是所有患者都能很好地耐受。该研究涵盖三个相互交织的主旨。第一个主旨将探索如何使用深度学习来解码由颅内深度和表面电极捕获的各种神经信号产生的语音。这一推动力将在多个患者体内和跨多个患者开发模型，这些模型可以稳健地解码语音，同时克服该领域当前的限制，从而有可能集成到语音神经假体中。第二个重点将探索有效的算法，用于根据记录的信号估计大脑连接动态。这一推动力将开发估计大量记录位点之间的定向连接的新技术，这对于理解认知处理过程中跨皮层的动态相互作用至关重要。第三个重点将开发深度学习模型，该模型可以仅根据神经记录来预测对语言处理至关重要的大脑区域。这些模型将有助于查明对语言处理至关重要的大脑区域，而无需对大脑进行电刺激。总而言之，拟议的研究利用深度学习领域的最新创新（例如变压器、图神经网络、自监督学习）来克服患者电极的非结构化和多样化放置、数据稀缺、可扩展性和稳定性带来的挑战。这些新方法将作为开源和可复制技术与科学界共享。该项目由计算神经科学合作研究 (CRCNS) 计划和残疾与康复工程计划 (DARE) 联合资助。该奖项反映了 NSF 的法定使命，并通过利用基金会的智力优势和更广泛的影响进行评估，认为值得支持审查标准。