RI: Small: Advancing the Science of Generalizable and Personalizable Speech-Centered Self-Report Emotion Classifiers

RI：小：推进以语音为中心的可概括和个性化的自我报告情绪分类器的科学

• 批准号: 2230172
• 负责人: Emily Provost
• 金额: $ 60万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2230172&HistoricalAwards=false
• 关键词: RI; Small; Advancing; Science; Generalizable; RI; Small; Advancing; Science; Generalizable

The goal of the project is to create new and personalized speech emotion recognition approaches and to use these approaches to investigate how changes in emotion are related to changes in mental health. The first step is accurately measuring how a person’s emotions vary over the course of a day, a week, a month, or even a year. However, the only approaches currently available to do so involve actively asking a user how they feel multiple times per day. Users are often willing to do this over shorter periods of time, but over longer periods of time this can be quite taxing. Fortunately, speech data are often easy to capture and conveys information about emotion. However, most approaches in speech emotion recognition are not focused on how the user feels and instead are focused on predicting how an outside group of people would label that user’s feeling. The goal of the project is to refocus automatic emotion classification on the user themselves. In the future, this will allow us to easily collect information about a user’s emotion leading to new investigations into how changes in emotions are associated with risk factors for changes in health.The goal of the presented research objectives is to advance the state-of-the-art in robust and generalizable personalized speech (acoustics + language) self-report emotion recognition classifiers and to investigate how measures created using these classifiers will allow researchers to intuit changes in mental health symptom severity in a clinical population of individuals at risk for suicidality. The field of automatic speech emotion recognition is almost exclusively focused on estimating how an outside group of observers would perceive a given emotional display (i.e., perception-of-other). Yet, when the focus is on the ultimate use cases of this technology, e.g., mental health symptom severity tracking, this is often not what is needed. Instead, symptom severity tracking often needs information about how a given individual is interpreting their own emotional experiences (i.e., self-report). For example, changing patterns in self-report are associated with changes in depression severity. Yet, these changes are currently measurable only through active participation, in which individuals are regularly asked to describe their emotional experiences using self-report measures (e.g., Ecological Momentary Assessment, EMA) longitudinally, multiple times per day, which can be quite expensive both in terms of cost and participant burden. The project team envisions a future in which audio can be passively collected and used to automatically infer self-reported emotion, but there has been limited attention to the design of such classifiers due to persistent challenges associated with accurately estimating self-reported emotion, including cognitive bias, context, and the difference between self-report and emotional experiences. The project team will accomplish these goals by: 1) creating classifiers that are robust and generalizable using new metrics that encourage models to attend to the same acoustic and language cues as human observers; 2) personalizing classifiers to users longitudinally, and 3) evaluating the effectiveness of self-report emotion classifiers by predicting changes in mental health symptom severity using an existing real-world dataset annotated with mental health symptom severity (risk of suicide). The presented approaches will forward investigations into how to use passively collected audio data to estimate changes in risk factors for health changes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的目的是创建新的和个性化的语音情感识别方法，并使用这些方法来研究情绪的变化与心理健康的变化如何相关。第一步是准确地衡量一个人的情绪在一天，一周，一个月甚至一年的过程中的变化。但是，目前唯一可用的方法会积极询问用户每天多次的感觉。用户通常愿意在较短的时间内执行此操作，但是在更长的时间内，这可能会很征一。幸运的是，语音数据通常很容易捕获和传达有关情绪的信息。但是，语音情感识别中的大多数方法并不集中于用户的感觉，而是专注于预测外部人群如何标记该用户的感觉。该项目的目的是将自动情感分类重新针对用户本身。将来，这将使我们能够轻松地收集有关用户情绪的信息，从而导致新投资与健康变化有关的风险因素如何相关。提出的研究目标的目的是在稳健而可及的个性化语音和可推广的个性化语音（Acoustics +语言 +语言）中促进自我报告情绪识别者的自我识别范围，并允许对这些分类者进行构成型号的构成型号，以促进稳健性的个性化语音（Acoustics +语言），以进行稳定的性格化和概括性，并允许对这些型号进行构成的范围。有自杀风险的人。自动语音情感识别领域几乎完全专注于估计外部观察者如何感知给定的情绪表现（即其他感知）。但是，当重点放在该技术的最终用例上时，例如心理健康症状严重性跟踪，这通常不是需要的。取而代之的是，症状严重性跟踪通常需要有关给定个人如何解释自己的情感经历（即自我报告）的信息。例如，自我报告中的变化模式与抑郁严重程度的变化有关。然而，这些变化目前只能通过积极参与来衡量，在这种情况下，经常要求个人使用自我报告措施（例如，生态瞬时评估，EMA）来描述自己的情感经历，每天多次，这在成本和参与伯伦方面都可能非常昂贵。该项目团队设想了一个未来，可以被动地收集音频，并用于自动推断自我报告的情绪，但是由于与准确估计自我报告的情绪相关的持续挑战，包括认知偏见，上下文以及自我报道和情感体验之间的差异，因此对这种分类器的设计有限。项目团队将通过以下方式实现这些目标：1）使用新指标创建可靠和推广的分类器，这些新指标鼓励模型与人类观察者一起参与相同的声学和语言提示； 2）纵向向用户个性化分类器，以及3）通过使用现有的现有现实世界中的数据集预测心理健康症状严重程度的变化来评估自我报告情绪分类器的有效性，并以心理健康症状严重程度（自杀风险）。提出的方法将对如何使用被动收集的音频数据进行转发调查以估计健康变化的风险因素的变化。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响来评估，被认为是珍贵的支持。