RI: Small: Collaborative Research: Exploring Audiovisual Emotion Perception using Data-Driven Computational Modeling

RI：小型：协作研究：使用数据驱动的计算模型探索视听情感感知

• 批准号: 1217183
• 负责人: Emily Provost
• 金额: $ 24.84万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2014-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1217183&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Exploring

This project explores perception-driven models of human audio-visual emotion using statistical analyses, data-driven computational modeling, and implicit sensing. Emotion underlies and modulates human communication. It is used in the diagnosis of many mental health conditions and is tracked in therapeutic interventions. Research in emotion perception seeks to identify models that describe the felt sense of 'typical' emotion expression -- i.e., an observer/evaluator's attribution of the emotional state of the speaker. This felt sense is a function of the methods through which individuals integrate the presented multi-modal emotional information. However, the nature of the interaction of the multi-modal cues is still an open question. This project will investigate multi-modal cue integration by studying how emotional inconsistency affects human perceptual judgment. In pursuit of this goal, the research objectives of this proposal are (1) to identify and validate primary and secondary audio-visual cues responsible for emotion perception, (2) to create a data-driven model to automatically predict the emotion perception of an evaluator, and (3) to predict evaluator state using implicit physiological and body gesture cues. The first research objective addresses the open question of how distal cues, the encoding of a speaker's communicative goals, interact and result in the felt sense of specific emotion states. Novel techniques will be used to identify emotionally salient distal cues using emotionally consistent and inconsistent audio-visual information. This identification has implications in the design of emotion classification algorithms and the emotional behavior of affective agents. The second research thrust addresses the open question of how human-centered models (rather than data-driven models) can be designed for use in emotion classification tasks. The project will investigate the efficacy of novel dynamic structures to model emotionally inconsistent information. These new structures will provide insights into the development of human-centered emotion classification inspired by the emotion perception process, rather than solely on data fluctuations. The third research objective addresses the open question regarding the effect of audio-visual emotion evaluation tasks on the evaluator's internal state. We will assess evaluator inattention in the context of emotional evaluation tasks. Models that can accurately predict evaluator inattention have applications in long-term human-computer and human-robot interaction platforms. The insights gained from this project will facilitate the design of emotion-focused algorithms that replicate the process by which humans interpret and integrate emotional audiovisual signals. It will also aid in the creation of emotional interfaces for health informatics applications, which will lead to more specifically targeted interventions and treatments for many mental health conditions including schizophrenia, depression, and autism.

该项目使用统计分析，数据驱动的计算建模和隐式感知探索了人类视听情绪的感知模型。 情绪是基础并调节人类交流。 它用于诊断许多精神健康状况，并在治疗干预措施中进行了跟踪。 情感感知的研究旨在识别描述“典型”情绪表达感觉的模型，即观察者/评估者对说话者情绪状态的归因。 这种感觉感是个人整合提出的多模式情感信息的方法的函数。 但是，多模式线索相互作用的性质仍然是一个悬而未决的问题。 该项目将通过研究情感上的不一致如何影响人类感知判断的方式来研究多模式提示整合。 为了实现这一目标，该提案的研究目标是（1）识别和验证负责情绪感知的主要和次要音频提示，（2）创建一个数据驱动的模型，以自动预测评估者的情绪感知，以及（3），以使用隐式的生理学和身体和身体和身体遗传来预测评估者状态。第一个研究的目的是解决远端提示，说话者交流目标的编码，互动并导致特定情感状态的感觉的开放问题。 新颖的技术将使用情感一致且不一致的视听信息来识别情感上的远端远端线索。 这种识别对情绪分类算法的设计和情感推动者的情感行为具有影响。 第二项研究推力解决了如何将以人为中心的模型（而不是数据驱动的模型）设计用于情绪分类任务的开放问题。 该项目将研究新型动态结构的功效，以模拟情感上不一致的信息。 这些新结构将提供有关受情感感知过程启发的以人为中心的情感分类发展的见解，而不仅仅是数据波动。 第三项研究目的是解决有关视听情绪评估任务对评估者内部状态的影响的开放问题。 我们将在情感评估任务的背景下评估评估者注意力不集中。 可以准确预测评估者注意力不集中的模型在长期的人类计算机和人类机器人相互作用平台中有应用。 从该项目中获得的见解将有助于设计以情感为中心的算法，这些算法复制了人类解释和整合情感视听信号的过程。 它还将有助于为健康信息学应用创建情感界面，这将导致针对许多精神分裂症，抑郁症和自闭症等许多心理健康状况的更具体针对性的干预措施和治疗。