CAREER: Automatic Speech-Based Longitudinal Emotion and Mood Recognition for Mental Health Monitoring and Treatment

职业:基于语音的自动纵向情感和情绪识别,用于心理健康监测和治疗

基本信息

项目摘要

Effective treatment and monitoring for individuals with mental health disorders is an enduring societal challenge. Regular monitoring increases access to preventative treatment, but is often cost prohibitive or infeasible given high demands placed on health care providers. Yet, it is critical for individuals with Bipolar Disorder (BPD), a chronic psychiatric illness characterized by mood transitions between healthy and pathological states. Transitions into pathological states are associated with profound disruptions in personal, social, vocational functioning, and emotion regulation. This Faculty Early Career Development Program (CAREER) project investigates new approaches in speech-based mood monitoring by taking advantage of the link between speech, emotion, and mood. The approach includes processing data with short-term variation (speech), estimating mid-term variation (emotion), and then using patterns in emotion to recognize long-term variation (mood). The educational outreach includes a design challenge, created with Iridescent, a science education nonprofit, that teaches emotion recognition to underserved children and their parents in informal learning settings. The research investigates methods to model naturalistic, longitudinal speech data and associate emotion patterns with mood, addressing current challenges in speech emotion recognition and assistive technology that include: generalizability, robustness, and performance. The approaches generalize to conditions whose symptoms include atypical emotion, such as post-traumatic stress disorder, anxiety, depression, and stress. The research forwards emotion as an intermediate step to simplify the mapping between speech and mood; emotion dysregulation is a common BPD symptom. Emotion is quantified over time in terms of valence and activation to improve generalizability. Nuisance modulations are controlled to improve robustness. Together, they result in a set of low-dimensional secondary features whose variations are due to emotion. These secondary features are segmented to create a coarser temporal description of emotion. This provides a means to map between speech (a quickly varying signal) and user state (a slowly varying signal), advancing the state-of-the-art. The results provide quantitative insight into the relationship between emotion variation and user state variation, providing new directions and links between the fields of emotion recognition and assistive technology. The focus on modeling emotional data using time series techniques results in breakthroughs in the design of emotion recognition and assistive technology algorithms.
对患有精神疾病的人的有效治疗和监测是一个持久的社会挑战。定期监测会增加获得预防治疗的机会,但考虑到对医疗保健提供者的高需求,通常是成本高昂或不可行的。然而,这对于患有躁郁症(BPD)的个体是至关重要的,这是一种以健康和病理状态之间情绪转变为特征的慢性精神病。向病理状态的过渡与个人,社会,职业功能和情绪调节的严重破坏有关。这个教师早期职业发展计划(职业)项目通过利用语音,情感和情绪之间的联系来调查基于语音的情绪监测的新方法。该方法包括处理短期变化(语音),估计中期变化(情绪)的处理数据,然后使用情感模式来识别长期变化(情绪)。教育外展活动包括由科学教育非营利组织Iridescent创建的设计挑战,该挑战在非正式的学习环境中向服务不足的儿童及其父母讲授情感认可。该研究调查了对自然主义,纵向语音数据进行建模的方法,并将情绪模式与情绪相关联,以应对语音情感识别和辅助技术的当前挑战,其中包括:概括性,鲁棒性和性能。 这些方法推广到症状包括非典型情绪的疾病,例如创伤后应激障碍,焦虑,抑郁和压力。该研究将情感作为中间步骤,以简化语音和情绪之间的映射。情绪失调是常见的BPD症状。随着时间的流逝,情绪在价值和激活方面被量化以提高普遍性。 滋扰调制受到控制以提高鲁棒性。 它们一起产生了一组低维二级特征,其变化是由于情感引起的。 这些次要特征被细分以创建对情感的更粗糙的时间描述。 这提供了一种在语音(快速变化的信号)和用户状态(缓慢变化的信号)之间映射的手段,从而推进了最新的。 结果提供了对情绪变化与用户状态变化之间关系的定量见解,从而提供了情感识别和辅助技术领域之间的新方向和联系。 使用时间序列技术对情绪数据进行建模的重点导致在情感识别和辅助技术算法的设计中取得突破。

项目成果

期刊论文数量(17)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Read speech voice quality and disfluency in individuals with recent suicidal ideation or suicide attempt
阅读最近有自杀意念或自杀企图的个人的语音质量和不流畅性
  • DOI:
    10.1016/j.specom.2021.05.004
  • 发表时间:
    2021
  • 期刊:
  • 影响因子:
    3.2
  • 作者:
    Stasak, Brian;Epps, Julien;Schatten, Heather T.;Miller, Ivan W.;Provost, Emily Mower;Armey, Michael F.
  • 通讯作者:
    Armey, Michael F.
Self-Ensembling Attention Networks: Addressing Domain Shift for Semantic Segmentation
  • DOI:
    10.1609/aaai.v33i01.33015581
  • 发表时间:
    2019-07
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Yonghao Xu;Bo Du;Lefei Zhang;Qian Zhang;Guoli Wang;Liangpei Zhang
  • 通讯作者:
    Yonghao Xu;Bo Du;Lefei Zhang;Qian Zhang;Guoli Wang;Liangpei Zhang
Into the Wild: Transitioning from Recognizing Mood in Clinical Interactions to Personal Conversations for Individuals with Bipolar Disorder
  • DOI:
    10.21437/interspeech.2019-2698
  • 发表时间:
    2019-01-01
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Matton, Katie;McInnis, Melvin G.;Provost, Emily Mower
  • 通讯作者:
    Provost, Emily Mower
Exploiting Acoustic and Lexical Properties of Phonemes to Recognize Valence from Speech
Cross-Corpus Acoustic Emotion Recognition with Multi-Task Learning: Seeking Common Ground While Preserving Differences
  • DOI:
    10.1109/taffc.2017.2684799
  • 发表时间:
    2019
  • 期刊:
  • 影响因子:
    11.2
  • 作者:
    Biqiao Zhang;E. Provost;Georg Essl
  • 通讯作者:
    Biqiao Zhang;E. Provost;Georg Essl
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Emily Provost其他文献

Emily Provost的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Emily Provost', 18)}}的其他基金

RI: Small: Advancing the Science of Generalizable and Personalizable Speech-Centered Self-Report Emotion Classifiers
RI:小:推进以语音为中心的可概括和个性化的自我报告情绪分类器的科学
  • 批准号:
    2230172
  • 财政年份:
    2022
  • 资助金额:
    $ 54.88万
  • 项目类别:
    Standard Grant
RI: Small: Speech-Centered Robust and Generalizable Measurements of "In the Wild" Behavior for Mental Health Symptom Severity Tracking
RI:小:以语音为中心的稳健且可概括的“野外”行为测量,用于心理健康症状严重程度跟踪
  • 批准号:
    2006618
  • 财政年份:
    2020
  • 资助金额:
    $ 54.88万
  • 项目类别:
    Standard Grant
A Workshop for Young Female Researchers in Speech Science and Technology
语音科学与技术领域年轻女性研究人员研讨会
  • 批准号:
    1835284
  • 财政年份:
    2018
  • 资助金额:
    $ 54.88万
  • 项目类别:
    Standard Grant
WORKSHOP: Doctoral Consortium at the International Conference on Multimodal Interaction (ICMI 2016)
研讨会:多模式交互国际会议上的博士联盟 (ICMI 2016)
  • 批准号:
    1641044
  • 财政年份:
    2016
  • 资助金额:
    $ 54.88万
  • 项目类别:
    Standard Grant
RI: Small: Collaborative Research: Exploring Audiovisual Emotion Perception using Data-Driven Computational Modeling
RI:小型:协作研究:使用数据驱动的计算模型探索视听情感感知
  • 批准号:
    1217183
  • 财政年份:
    2012
  • 资助金额:
    $ 54.88万
  • 项目类别:
    Continuing Grant

相似国自然基金

基于基因组数据自动化分析为后生动物类群大规模开发扩增子捕获探针的实现
  • 批准号:
    32370477
  • 批准年份:
    2023
  • 资助金额:
    50 万元
  • 项目类别:
    面上项目
基于非驾驶姿态多维特征的自动驾驶接管风险态势辨识与自适应调控策略
  • 批准号:
    52372325
  • 批准年份:
    2023
  • 资助金额:
    49 万元
  • 项目类别:
    面上项目
面向生物医学文本的知识自动总结研究
  • 批准号:
    72304189
  • 批准年份:
    2023
  • 资助金额:
    30 万元
  • 项目类别:
    青年科学基金项目
自动机序列的密码安全性分析
  • 批准号:
    62372256
  • 批准年份:
    2023
  • 资助金额:
    50 万元
  • 项目类别:
    面上项目
基于国产AI芯片的自动布局布线优化算法研究
  • 批准号:
    62306286
  • 批准年份:
    2023
  • 资助金额:
    30 万元
  • 项目类别:
    青年科学基金项目

相似海外基金

CAREER: Breaking the phonetic code: novel acoustic-lexical modeling techniques for robust automatic speech recognition
职业:打破语音密码:用于鲁棒自动语音识别的新颖声学词汇建模技术
  • 批准号:
    0643901
  • 财政年份:
    2006
  • 资助金额:
    $ 54.88万
  • 项目类别:
    Continuing Grant
CAREER: Adaptive and Robust Automatic Speech Recognition inHuman-Computer Interaction
职业:人机交互中的自适应和鲁棒自动语音识别
  • 批准号:
    9996042
  • 财政年份:
    1998
  • 资助金额:
    $ 54.88万
  • 项目类别:
    Continuing Grant
University of Utah Biomedical Informatics Training
犹他大学生物医学信息学培训
  • 批准号:
    7214258
  • 财政年份:
    1997
  • 资助金额:
    $ 54.88万
  • 项目类别:
University of Utah Medical Informatics Training
犹他大学医学信息学培训
  • 批准号:
    7337515
  • 财政年份:
    1997
  • 资助金额:
    $ 54.88万
  • 项目类别:
University of Utah Biomedical Informatics Training
犹他大学生物医学信息学培训
  • 批准号:
    7457689
  • 财政年份:
    1997
  • 资助金额:
    $ 54.88万
  • 项目类别:
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了