Computational phenotyping of face expression in early psychosis

早期精神病面部表情的计算表型

• 批准号: 10608718
• 负责人: CHERYL MARY CORCORAN
• 金额: $ 70.65万
• 依托单位: ICAHN SCHOOL OF MEDICINE AT MOUNT SINAI
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-04-15 至 2027-01-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10608718
• 关键词: Acoustics; Address; Affect; Age; Anatomy; Archives; Australia; Behavior; Behavior assessment; Behavioral; Biological Markers; Canada; Characteristics; Chronic; Clinical; Clinical Trials; Code; Cognition; Cognitive; Companions; Complex; Computer Analysis; Computing Methodologies; Data; Development; Electromyography; Ethnic Origin; Evidence based treatment; Expressed Emotion; Eye; Face; Face Processing; Facial Expression; Funding; Genetic Transcription; Hearing; Image; Impairment; Individual; International; Interview; Interviewer; Label; Language; Longitudinal Studies; Measures; Methodological Studies; Modality; Modeling; Movement; Muscle; National Institute of Mental Health; Natural Language Processing; Outcome; Participant; Patients; Pattern; Persuasive Communication; Pharmaceutical Preparations; Phenotype; Physiological; Physiology; Property; Psychometrics; Psychoses; Research Domain Criteria; Risk; Role; Schizophrenia; Series; Signal Transduction; Smiling; Speech; Stimulus; Stream; Symptoms; Testing; Time; Time Series Analysis; Variant; Voice; Work; automated analysis; clinical high risk for psychosis; cohort; deep neural network; demographics; design; early psychosis; feature extraction; follow up assessment; functional outcomes; gaze; high risk; indexing; multimodal data; multimodality; non-verbal; novel; processing speed; schizophrenia spectrum disorder; sex; social; syntax

PROJECT SUMMARY/ABSTRACT Blunted or flat facial expression is characteristic of schizophrenia (Sz) spectrum disorders and their clinical high risk (CHR) states, and is associated with negative symptoms, social impairment, and poor outcome. Currently, there is no objective test to quantify blunted face expression in Sz. However, with advancements in computational methods, we can begin to operationalize blunted face expression, necessary for both methodological studies and clinical trials, especially as there are no evidence-based treatments. We propose computational analyses of time series of video frame-based estimates of movements of individual face muscles, based on Ekman and Friesen’s Facial Action Coding System or FACS, obtained during ecologically valid open-ended interview on a remote platform (and during a standard viewing paradigm of valenced stimuli). Face action units (AUs) index movement of individual face muscles, which have known physiology and circuitry. Our preliminary data are consistent with early small video coding and electromyography studies of face AUs in Sz, in replicating decreased mean amplitude of face AUs, especially of AU7 (“lid tightener”) or orbicularis oculi, which is involved in smiling, but also social signaling more broadly. Further, in generating matrix and distance profiles for face AU time series data, we find indices of decreased repertoire of face expression in Sz, also associated like amplitudes in CHR with ratings of blunted affect and poor function. Aim 1 is to assess the correlates and psychometric properties of these face expression amplitude and repertoire metrics in a large international cohort of early course Sz spectrum, CHR and healthy individuals well-characterized for demographics, symptoms, cognition and function, (and in whom effects of medications and chronicity may be less). These metrics are expected to differentiate psychosis from the norm, have convergent validity with ground truth ratings of “blunted affect”, and be correlated with social and role impairment, and also face processing ability. Variation by demographics (age, sex, ethnicity) will be assessed, as well as test-retest reliability at one year. Aim 2 is to align video and audio time series to create per frame sets of features and envelope metrics in segments in participants to test for differences in facial dynamics when individuals are speaking vs. listening. We hypothesize that global face expression will decrease significantly during pauses in individuals with Sz, consistent with preliminary data. Based on a cognitive model, we hypothesize decreased face expression and pause behavior will be correlated and associated with slowed processing speed. Aim 3 assesses synchrony of face expression between interviewers and participants, with patients hypothesized to have decreased synchrony and alignment across modalities. Overall, this large rich dataset of multimodal time series raw data will be archived and available for analyses, including more complex nonlinear time series analyses.

项目概要/摘要 面部表情迟钝或平淡是精神分裂症 (Sz) 谱系障碍的特征，其临床高度 目前，风险（CHR）状态与阴性症状、社交障碍和不良结果相关。 然而，随着计算技术的进步，没有客观的测试来量化 Sz 中迟钝的面部表情。 方法，我们可以开始操作迟钝的面部表情，这对于两种方法论研究都是必要的 和临床试验，特别是因为没有基于证据的治疗方法，我们建议进行计算分析。 基于视频帧的个体面部肌肉运动估计的时间序列，基于 Ekman 和 弗里森的面部动作编码系统或 FACS，在生态有效的开放式访谈中获得 远程平台（以及在有价刺激的标准观看范例期间）面部动作单位（AU）索引。 各个面部肌肉的运动，我们的初步数据是已知的生理学和电路。 与 Sz 中面部 AU 的早期小视频编码和肌电图研究一致，复制减少 面部 AU 的平均振幅，尤其是与微笑有关的 AU7（“眼睑收紧器”）或眼轮匝肌的平均振幅， 此外，还包括更广泛的社交信号，用于生成人脸 AU 时间序列的矩阵和距离配置文件。 数据中，我们发现 Sz 中面部表情下降的指数，也与 CHR 中的振幅相关 目标 1 是评估情感迟钝和功能差的相关性和心理测量特性。 这些面部表情幅度和曲目指标是在早期课程 Sz 的大型国际队列中进行的 谱、CHR 和健康个体的人口统计、症状、认知和功能， （并且药物和慢性病对哪些人的影响可能较小）。这些指标预计会有所不同。 与正常的精神病相比，与“钝化情感”的基本事实评级具有收敛有效性，并且具有相关性 具有社交和角色障碍，并且面部处理能力也因人口统计（年龄、性别、种族）而变化。 将进行评估，以及一年后的重测可靠性。目标 2 是使视频和音频时间序列保持一致。 创建每帧的特征集和分段中的包络指标，以测试面部差异 当个人说话与倾听时的动态变化我们勇敢地面对整体面部表情会减少的情况。 根据初步数据，Sz 患者的停顿期间显着。 我们面临面部表情减少，暂停行为将与速度减慢相关联 目标 3 评估访谈者和参与者之间面部表情的同步性。 总体而言，患者之间的同步性和一致性有所下降。 多模式时间序列原始数据的数据集将被存档并可用于分析，包括更复杂的分析 非线性时间序列分析。