NCS-FO: Brain-Informed Goal-Oriented and Bidirectional Deep Emotion Inference

NCS-FO：大脑知情的目标导向双向深度情感推理

• 批准号: 2318984
• 负责人: Ruogu Fang
• 金额: $ 92万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2318984&HistoricalAwards=false
• 关键词: NCS; FO; Brain; Informed; Goal

Human emotions are dynamic, multidimensional responses to challenges and opportunities that emerge from network interactions in the brain. Disruptions of these dynamics underlie emotional dysregulation in many mental disorders including anxiety and depression. To empirically study the neural basis of human emotion inference, experimenters often have observers view natural images varying in affective content, while at the same time recording their brain activity using electroencephalogram (EEG) and/or Functional magnetic resonance imaging (fMRI). Despite extensive research over the last few decades, much remains to be learned about the computational principles subserving the recognition of emotions in natural scenes. A major roadblock faced by empirical neuroscientists is the inability to carry out precisely manipulate human neural systems and test the consequences in imaging data. Deep Neural Networks (DNN), owing to their high relevance to human neural systems and extraordinary prediction capability, have become a promising tool for testing these sorts of hypotheses in swift and nearly costless computer simulations. The overarching goal of this project is to develop a neuroscience-inspired, DNN-based deep learning framework for emotion inference in real-world scenarios by synergistically integrating neuron-, circuit-, and system-level mechanisms. Recognizing that the state-of-the-art DNNs are centered on bottom-up and feedforward-only processing, which disagrees with the strong goal-oriented top-down modulation recurrence observed in the physiology, this project aims to enrich DNNs and enable closer AI-neuroscience interaction by incorporating goal-oriented top-down modulation and reciprocal interactions DNNs and test the model assumptions and predictions on neuroimaging data.To meet these goals, the project aims to develop a brain-inspired goal-oriented and bidirectional deep learning model for emotion inference. Despite the great promise shown by today’s deep learning as a framework for modeling biological vision, their architecture is limited to emulating the visual cortex for face/object/scene recognition and rarely goes beyond the inferotemporal cortex (IT), which is necessary for modeling high-level cognitive processes. In this project, we propose to build a biologically plausible deep learning architecture by integrating an in-silico amygdala module into the visual cortex architecture in DNN (the VCA model). The researchers hope to build neuron-, circuit-, and system-level modulation via goal-oriented attention priming, and multi-pathway predictive coding to 1) elucidate the mechanism of selectivity underlying preference and response to naturalistic emotions by artificial neurons; 2) differentiate fine-grained emotional responses via multi-path predictive coding, and 3) refine the neuroscientific understanding of human neuro-behavioral data by comparing attention priming and temporal generalization observed in simultaneous fMRI-EEG data to the computational observations using our brain-inspired VCA model. This project introduces two key innovations, both patterned after how brain operates, into DNN architecture and demonstrate their superior performance when applied to complex real-world tasks. Successful execution of the project can lead to the development of a new generation of AI-models that are inspired by neuroscience and that may in turn power neuroscience research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类情绪是对大脑网络交互中出现的挑战和机遇的动态、多维反应，这些动态的破坏是包括焦虑和抑郁在内的许多精神疾病的情绪失调的基础。观察者观看情感内容各异的自然图像，同时使用脑电图（EEG）和/或功能磁共振成像（fMRI）记录他们的大脑活动，尽管在过去的几十年里进行了广泛的研究，但仍有很多东西有待了解。计算性的经验神经科学家面临的一个主要障碍是无法精确操纵人类神经系统并测试成像数据中的结果，因为它们与人类高度相关。神经系统和非凡的预测能力，已成为在快速且几乎无成本的计算机模拟中测试此类假设的有前景的工具。该项目的总体目标是开发一种受神经科学启发、基于 DNN 的深度学习框架，用于情感推理。认识到最先进的 DNN 以自下而上和仅前馈处理为中心，这与强烈的目标导向不同。生理学中观察到的自上而下的调制循环，该项目旨在通过结合目标导向的自上而下的调制和相互交互的 DNN 来丰富 DNN 并实现更紧密的 AI 与神经科学交互，并测试模型假设和预测为了实现这些目标，该项目旨在开发一种受大脑启发的、面向目标的双向深度学习模型，用于情感推理。仅限于模拟视觉皮层进行面部/物体/场景识别，很少超越颞下皮层（IT），而下颞叶皮层是建模高级认知过程所必需的。在这个项目中，我们建议通过以下方式构建生物学上合理的深度学习架构。一个整合的研究人员希望通过目标导向的注意力启动和多路径预测编码来构建神经元、电路和系统级调制，以实现 1)。阐明人工神经元对自然情绪的偏好和反应的选择性机制；2）通过多路径预测编码区分细粒度的情绪反应，3）完善对人类的神经科学理解通过将同步 fMRI-EEG 数据中观察到的注意力启动和时间泛化与使用我们的受大脑启发的 VCA 模型进行的计算观察进行比较，我们获得了神经行为数据。该项目将两项关键创新（均以大脑的运作方式为模式）引入了 DNN 架构，并展示了它们的应用。该项目的成功执行可以促进受神经科学启发的新一代人工智能模型的开发，从而为神经科学研究提供动力。该奖项反映了美国国家科学基金会的法定奖项。使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。