NCS-FO: Brain-Informed Goal-Oriented and Bidirectional Deep Emotion Inference

NCS-FO：大脑知情的目标导向双向深度情感推理

• 批准号: 2318984
• 负责人: Ruogu Fang
• 金额: $ 92万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2318984&HistoricalAwards=false
• 关键词: NCS; FO; Brain; Informed; Goal

Human emotions are dynamic, multidimensional responses to challenges and opportunities that emerge from network interactions in the brain. Disruptions of these dynamics underlie emotional dysregulation in many mental disorders including anxiety and depression. To empirically study the neural basis of human emotion inference, experimenters often have observers view natural images varying in affective content, while at the same time recording their brain activity using electroencephalogram (EEG) and/or Functional magnetic resonance imaging (fMRI). Despite extensive research over the last few decades, much remains to be learned about the computational principles subserving the recognition of emotions in natural scenes. A major roadblock faced by empirical neuroscientists is the inability to carry out precisely manipulate human neural systems and test the consequences in imaging data. Deep Neural Networks (DNN), owing to their high relevance to human neural systems and extraordinary prediction capability, have become a promising tool for testing these sorts of hypotheses in swift and nearly costless computer simulations. The overarching goal of this project is to develop a neuroscience-inspired, DNN-based deep learning framework for emotion inference in real-world scenarios by synergistically integrating neuron-, circuit-, and system-level mechanisms. Recognizing that the state-of-the-art DNNs are centered on bottom-up and feedforward-only processing, which disagrees with the strong goal-oriented top-down modulation recurrence observed in the physiology, this project aims to enrich DNNs and enable closer AI-neuroscience interaction by incorporating goal-oriented top-down modulation and reciprocal interactions DNNs and test the model assumptions and predictions on neuroimaging data.To meet these goals, the project aims to develop a brain-inspired goal-oriented and bidirectional deep learning model for emotion inference. Despite the great promise shown by today’s deep learning as a framework for modeling biological vision, their architecture is limited to emulating the visual cortex for face/object/scene recognition and rarely goes beyond the inferotemporal cortex (IT), which is necessary for modeling high-level cognitive processes. In this project, we propose to build a biologically plausible deep learning architecture by integrating an in-silico amygdala module into the visual cortex architecture in DNN (the VCA model). The researchers hope to build neuron-, circuit-, and system-level modulation via goal-oriented attention priming, and multi-pathway predictive coding to 1) elucidate the mechanism of selectivity underlying preference and response to naturalistic emotions by artificial neurons; 2) differentiate fine-grained emotional responses via multi-path predictive coding, and 3) refine the neuroscientific understanding of human neuro-behavioral data by comparing attention priming and temporal generalization observed in simultaneous fMRI-EEG data to the computational observations using our brain-inspired VCA model. This project introduces two key innovations, both patterned after how brain operates, into DNN architecture and demonstrate their superior performance when applied to complex real-world tasks. Successful execution of the project can lead to the development of a new generation of AI-models that are inspired by neuroscience and that may in turn power neuroscience research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类的情绪是对大脑网络互动所带来的挑战和机遇的动态，多维的反应。这些动态的破坏是许多精神障碍（包括焦虑和抑郁）的情绪失调的基础。为了紧急研究人类情绪推断的神经元基础，实验者通常会让观察者查看自然图像在情感含量上有所不同，同时使用脑电图（EEG）和/或功能磁共振成像（FMRI）记录其大脑活动。尽管在过去的几十年中进行了广泛的研究，但有关计算原则在自然场景中对情绪的认识的依据还有很多尚待了解。经验神经科学家面临的主要障碍是无法精确操纵人类神经系统并测试成像数据中的后果。深度神经网络（DNN）由于它们与人类神经系统的高度相关性和非凡的预测能力，已成为在迅速而几乎无价计算机模拟中测试这些假设的有望工具。该项目的总体目标是开发一个神经科学启发的，基于DNN的深度学习框架，以通过协同整合神经，电路和系统级别的机制在实际情况下进行情感推断。认识到最先进的DNN集中在自下而上和仅饲喂方面的处理上，这与生理学中观察到的强大的面向目标的自上而下的调节复发不同意，该项目旨在丰富DNN，并通过启用更接近的AI-Weuroscience互动，并通过模型进行启动的量子互动，并将其转换为模型的射击量表，并进行验证范围内的射击范围，以供射击量牌进行登录的调节，以实现端口的范围。神经影像学数据。为了实现这些目标，该项目旨在开发以脑为目标和双向深度学习模型来推断情绪推断。尽管当今的深度学习是建模生物学视觉的框架，但它们的架构仅限于模拟面部/对象/场景识别的视觉皮层，并且很少超越了上颞皮层（IT），这对于建模高级认知过程是必不可少的。在这个项目中，我们建议通过将silico Amygdala模块集成到DNN（VCA模型）的Visual Cortex体系结构中来构建生物学上合理的深度学习体系结构。研究人员希望通过以目标为导向的注意力启动来构建神经元，电路和系统级调制，并为1）阐明人工神经元对自然情绪的选择性和反应的选择性机制； 2）通过多路预测编码来区分细粒度的情绪响应，3）通过将注意力启动和在简单的FMRI-EEG数据中观察到的注意力启动和临时概括与使用我们的脑启发的VCA模型中观察到的注意力启动和临时概括，从而完善对人神经行为数据的神经科学理解。该项目介绍了两个关键的创新，均以大脑的运作方式进行图案，并将其应用于复杂的现实世界任务时展示其出色的性能。该项目的成功执行可能会导致新一代的AI模型的发展，这些AI模型受神经科学的启发，进而可能又可能是神经科学研究。这项奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响标准来评估NSF的法定任务。