Computational and brain predictors of emotion cue integration

情绪线索整合的计算和大脑预测因子

• 批准号: 9923725
• 负责人: Jamil Zaki
• 金额: $ 45.18万
• 依托单位: STANFORD UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-05-19 至 2022-02-28
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9923725
• 关键词: Affect; Agreement; Base of the Brain; Basic Science; Bayesian Modeling; Bipolar Disorder; Brain; Brain region; Classification; Cognitive; Complex; Computer Models; Cues; Data; Dependence; Emotional; Emotional disorder; Emotions; Event; Exhibits; Face; Face Processing; Facial Expression; Functional Magnetic Resonance Imaging; Future; Image; Individual; Language; Lateral; Learning; Life; Linguistics; Machine Learning; Major Depressive Disorder; Measures; Mental disorders; Methods; Modality; Modeling; Mood Disorders; Moods; Movement; National Institute of Mental Health; Neurodevelopmental Disorder; Neurosciences; Observer Variation; Participant; Patients; Pattern; Perception; Process; Psychological reinforcement; Psychologist; Research; Research Domain Criteria; Research Personnel; Running; Sampling; Scanning; Scientist; Sensory; Social Functioning; Social Interaction; Social Processes; Social Psychology; Specific qualifier value; Speech; Structure; System; Testing; Time; Training; Treatment Efficacy; Visual; Weight; Work; affective computing; base; brain abnormalities; cognitive neuroscience; computer science; computerized tools; executive function; insight; language comprehension; multimodality; neuroimaging; novel; recruit; relating to nervous system; response; scaffold; social; social deficits; tool

The purpose of this project is to develop computational and brain-based models of emotion cue integration: people’s inferences about others’ emotions based on dynamic, multimodal cues. Observers often decide how targets feel based on cues such as facial expressions, prosody, and language. Such inferences scaffold healthy social interaction, and abnormal inference both marks and exacerbates social deficits in numerous psychiatric disorders. Psychologists and neuroscientists have studied emotion inference for decades, but the vast majority of this work employs simplified social cues, such as vignettes or static images of faces. By contrast, “real world” emotion cues are complex, dynamic, and multimodal. Cue integration—inference based on naturalistic emotion information—likely differs from simpler inference at cognitive and neural levels, but this phenomenon remains poorly understood. This means that scientists lack a clear model of how observers adaptively process complex emotion cues, and how that processing goes awry in mental illness. Especially lacking are mechanistic models that can describe the computations and brain processes involved in cue integration with sufficient precision to predict inference in new cases, observers, and samples. This project will merge tools from social psychology, computer science, and neuroscience to generate a novel and rigorous model of emotion cue integration. We have demonstrated that in the face of complex emotion cues, observers dynamically “weight” cues from each modality (e.g., visual, linguistic) over time, a process that (i) tracks shifts in brain activity and connectivity; and (ii) can be captured using Bayesian models. Here, we will expand this work in several ways. First, we will develop precise computational tools to isolate features of emotion cues—such as facial movements, prosody, and linguistic sentiment—that track observers’ use of each cue modality during integration. Second, we will develop multi-region “signatures” of brain activity and connectivity that track emotion inference in each modality. We will use these signatures in conjunction with machine learning to predict unimodal emotion inference and cue integration in new observers and samples, based on brain data alone. Third, we will explore the context-dependence of naturalistic emotion inference by testing whether reinforcement learning can bias observers’ cue integration and accompanying brain signatures. Finally, we will model computational and neural abnormalities associated with cue integration in patients with Major Depressive Disorder and Bipolar Disorder. At the level of basic science, these data will generate a fundamentally new—and more naturalistic—approach to the neuroscience of emotion inference. The computational and brain metrics we produce will also be made publically available to facilitate the open and cumulative study of emotion inference across labs. At a translational level, we will provide a mechanistic, rich account of abnormal emotion inference in mood disorders, paving the way for computational and brain markers that can be used to assess social dysfunction and treatment efficacy in these and other mental illnesses.

该项目的目的是开发基于计算和大脑的情感提示整合模型： 人们根据动态的多模式提示对他人情绪的推论。观察者经常决定如何 目标基于面部表情，韵律和语言等线索的感觉。这样的推论脚手架 健康的社交互动，以及许多标记和加剧社会缺陷的异常推理 精神疾病。心理学家和神经科学家已经研究了几十年来的情感推断，但是 这些工作的绝大多数员工简化了社交线索，例如小插图或面孔的静态图像。经过 对比，“现实世界”情绪提示是复杂，动态和多模式的。提示集成 - 基于参与 关于自然情绪信息，与认知和神经层面上的简单推论完全不同，但这 现象仍然知之甚少。这意味着科学家缺乏明确的观察者的模型 适应性地处理复杂的情绪，以及该处理如何在精神疾病中出现问题。尤其 缺乏机械模型可以描述提示中涉及的计算和大脑过程 积分具有足够的精确度以预测新病例，观察者和样本的推论。这个项目将 合并社会心理学，计算机科学和神经科学的工具，以产生小说和 严格的情绪提示整合模型。我们已经证明，面对复杂的情绪 提示，观察者随着时间的流逝，从每种模态（例如，视觉，语言）中动态“权重”提示，这一过程 （i）跟踪大脑活动和连通性的变化； （ii）可以使用贝叶斯型号捕获。在这里，我们会的 通过几种方式扩展这项工作。首先，我们将开发精确的计算工具来隔离 情感提示，例如面部运动，韵律和语言情感 - 轨道观察者的使用 在整合过程中提示方式。其次，我们将开发大脑活动的多区域“签名”和 跟踪每种方式中情感推断的连接性。我们将与这些签名结合使用 机器学习以预测新观察者和样本中的单峰情绪推断和提示整合， 仅基于大脑数据。第三，我们将探讨自然主义情感推断的上下文依赖性 测试强化学习是否可以使观察者的提示整合和携带大脑签名偏见。 最后，我们将建模与与提示整合相关的计算和神经异常 重度抑郁症和躁郁症。在基础科学层面上，这些数据将产生 从根本上讲，对情绪推断的神经科学的新鲜感，更自然。这 我们生产的计算和大脑指标也将公开使用，以促进开放和 跨实验室的情绪推断的累积研究。在翻译层面，我们将提供一种机械，丰富的 情绪障碍中情绪异常推断的解释，为计算和大脑标记铺平了道路 可以用来评估这些和其他精神疾病的社会功能障碍和治疗效率。