NCS-FR: Engineering Brain Circuits for Complex Scene Analysis

NCS-FR：用于复杂场景分析的工程大脑电路

• 批准号: 2319321
• 负责人: Kamal Sen
• 金额: $ 296.19万
• 依托单位: Trustees of Boston University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2028-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2319321&HistoricalAwards=false
• 关键词: NCS; FR; Engineering; Brain; Circuits

Everyday social situations, like a crowded party, a restaurant, a classroom, or an open-plan workplaces, involve multiple speakers and listeners and the hum of background noise. In these complex sound environments, humans with typical hearing are able to identify and listening to individual sound sources, for example what a single speaker is saying, while ignoring the other sound sources, for example someone else's phone call or background noise, like cars driving down the street. This is an example of a general problem called complex scene analysis (CSA), and a full understanding of how humans with typical hearing solve this problem has remained elusive to scientists from a diverse range of fields - neuroscience, computer science, speech recognition and engineering - even after more than 50 years of research. Because of this, CSA remains a problem for many humans, like those with hearing impairment, for medical devices, like hearing aids, and for technology, for example automatic speech recognition systems. This project investigates the neural basis of complex scene analysis in typical hearing, and, based on these discoveries, develops a brain inspired algorithm for CSA. This project will ultimately improve quality of life through a variety of applications, for example for improving the effectiveness of hearing aids and speech recognition technologies. Solving this problem requires an interdisciplinary effort, and as part of the research, an educational platform is developed to train students to integrate knowledge from a variety of disciplines that makes them better able to address challenging and important societal problems. This project integrates three interdisciplinary research threads to develop the brain-inspired algorithm. The first thread uses brain imaging in humans performing CSA with an integrated wearable device that measures brain signals (functional near-infrared spectroscopy and electroencephalography), and machine learning methods to decode where a subject is attending in a complex audiovisual scene. The second thread investigates cortical circuits for CSA in attentive states, which are thought to enhance CSA performance. This thread integrates electrophysiology, optogenetics, behavior and computational modeling in mice, a model system with well-established, powerful experimental tools for unraveling cortical circuits. The third thread designs an attention steered algorithm for the wearable device that selectively processes an attended source in a complex scene, integrating the attended location decoded from a subject’s brain signals (thread 1), and a model of cortical circuits in attentive states (thread 2). This thread optimizes the algorithm to generate a fast, compact, energy efficient, and state of the art algorithm for CSA and evaluate its performance in humans.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

每天的社交场合，例如拥挤的聚会，餐厅，教室或开放式工作场所，都涉及多个扬声器和听众以及背景噪音的嗡嗡声。在这些复杂的声音环境中，具有典型听力的人能够识别和聆听单个声音来源，例如单个扬声器在说什么，同时忽略其他声音来源，例如别人的电话或背景噪音，例如在街上驾驶的汽车。这是一个称为复杂场景分析（CSA）的一般问题的一个例子，并且对典型听力的人的充分理解该问题仍然难以捉摸，即使经过超过50年的研究，科学家（神经科学，计算机科学，语音识别和工程学）的科学家仍然难以捉摸。因此，对于许多人来说，CSA仍然是一个问题，例如那些有听力障碍的人，对于助听器等医疗设备和技术，例如自动语音识别系统。该项目研究了典型听力中复杂场景分析的神经元基础，并根据这些发现开发了CSA的大脑启发算法。该项目最终将通过各种应用来改善生活质量，例如，以提高助听器和语音识别技术的有效性。解决这个问题需要跨学科的工作，作为研究的一部分，开发了一个教育平台来培训学生从各种学科中整合知识，使他们能够更好地解决挑战和重要的社会问题。该项目集成了三个跨学科研究线程，以开发受脑启发的算法。第一个线程在人类中使用大脑成像进行CSA，并具有集成的可穿戴设备，该设备可测量脑信号（功能性近红外光谱和脑电图），并使用机器学习方法来解码受试者在复杂的视听场景中参加的情况。第二个线程研究了细心状态下CSA的皮质回路，这被认为可以提高CSA性能。该线程集成了小鼠中的电生理学，光遗传学，行为和计算建模，这是一种模型系统，具有完善的，功能强大的实验工具，用于揭示皮质电路。第三个线程为可穿戴设备设计了一种注意力的算法，该算法在复杂的场景中有选择地处理所访问的源，集成了从受试者的大脑信号（线程1）和细心状态下的皮质电路模型中解码的所在位置（线程2）。该线程优化了算法，以生成快速，紧凑，能源效率和CSA的最先进算法的状态，并评估其在人类中的表现。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛影响的审查标准来评估，被认为是通过评估来获得的支持。