RI: Small: Collaborative Research: Exploring Audiovisual Emotion Perception using Data-Driven Computational Modeling

RI：小型：协作研究：使用数据驱动的计算模型探索视听情感感知

• 批准号: 1217183
• 负责人: Emily Provost
• 金额: $ 24.84万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2014-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1217183&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Exploring

This project explores perception-driven models of human audio-visual emotion using statistical analyses, data-driven computational modeling, and implicit sensing. Emotion underlies and modulates human communication. It is used in the diagnosis of many mental health conditions and is tracked in therapeutic interventions. Research in emotion perception seeks to identify models that describe the felt sense of 'typical' emotion expression -- i.e., an observer/evaluator's attribution of the emotional state of the speaker. This felt sense is a function of the methods through which individuals integrate the presented multi-modal emotional information. However, the nature of the interaction of the multi-modal cues is still an open question. This project will investigate multi-modal cue integration by studying how emotional inconsistency affects human perceptual judgment. In pursuit of this goal, the research objectives of this proposal are (1) to identify and validate primary and secondary audio-visual cues responsible for emotion perception, (2) to create a data-driven model to automatically predict the emotion perception of an evaluator, and (3) to predict evaluator state using implicit physiological and body gesture cues. The first research objective addresses the open question of how distal cues, the encoding of a speaker's communicative goals, interact and result in the felt sense of specific emotion states. Novel techniques will be used to identify emotionally salient distal cues using emotionally consistent and inconsistent audio-visual information. This identification has implications in the design of emotion classification algorithms and the emotional behavior of affective agents. The second research thrust addresses the open question of how human-centered models (rather than data-driven models) can be designed for use in emotion classification tasks. The project will investigate the efficacy of novel dynamic structures to model emotionally inconsistent information. These new structures will provide insights into the development of human-centered emotion classification inspired by the emotion perception process, rather than solely on data fluctuations. The third research objective addresses the open question regarding the effect of audio-visual emotion evaluation tasks on the evaluator's internal state. We will assess evaluator inattention in the context of emotional evaluation tasks. Models that can accurately predict evaluator inattention have applications in long-term human-computer and human-robot interaction platforms. The insights gained from this project will facilitate the design of emotion-focused algorithms that replicate the process by which humans interpret and integrate emotional audiovisual signals. It will also aid in the creation of emotional interfaces for health informatics applications, which will lead to more specifically targeted interventions and treatments for many mental health conditions including schizophrenia, depression, and autism.

该项目利用统计分析、数据驱动的计算模型和隐式感知来探索人类视听情感的感知驱动模型。 情感是人类交流的基础并调节着人类交流。 它用于诊断许多心理健康状况，并在治疗干预中进行跟踪。 情绪感知研究旨在确定描述“典型”情绪表达的感受的模型，即观察者/评估者对说话者情绪状态的归因。 这种感觉是个人整合所呈现的多模态情感信息的方法的函数。 然而，多模态线索相互作用的本质仍然是一个悬而未决的问题。 该项目将通过研究情绪不一致如何影响人类感知判断来研究多模式线索整合。 为了实现这一目标，本提案的研究目标是（1）识别和验证负责情绪感知的主要和次要视听线索，（2）创建一个数据驱动模型来自动预测一个人的情绪感知。 (3) 使用隐含的生理和身体姿势线索来预测评估者状态。第一个研究目标解决了一个悬而未决的问题，即远端线索（说话者交际目标的编码）如何相互作用并导致特定情绪状态的感受。 新技术将用于利用情感一致和不一致的视听信息来识别情感上显着的远端线索。 这种识别对情感分类算法的设计和情感主体的情感行为具有影响。 第二个研究重点解决了如何设计以人为中心的模型（而不是数据驱动的模型）以用于情感分类任务的悬而未决的问题。 该项目将研究新颖的动态结构对情感不一致信息进行建模的有效性。 这些新结构将为受情绪感知过程启发的以人为中心的情绪分类的发展提供见解，而不仅仅是数据波动。 第三个研究目标解决了有关视听情感评估任务对评估者内部状态的影响的悬而未决的问题。 我们将在情感评估任务的背景下评估评估者的注意力不集中。 能够准确预测评估者注意力不集中的模型在长期人机和人机交互平台中具有应用。 从该项目中获得的见解将有助于设计以情感为中心的算法，该算法复制人类解释和整合情感视听信号的过程。 它还将有助于为健康信息学应用创建情感界面，从而为包括精神分裂症、抑郁症和自闭症在内的许多心理健康状况提供更有针对性的干预和治疗。