SenSE: Artificial Intelligence-enabled Multimodal Stress Sensing for Precision Health

SenSE：人工智能支持的多模态压力传感，实现精准健康

• 批准号: 2037304
• 负责人: Zhenan Bao
• 金额: $ 75万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-15 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2037304&HistoricalAwards=false
• 关键词: SenSE; Artificial; Intelligence; enabled; Multimodal

A shared objective of precision health and psychiatry is to precisely measure relevant symptoms and sustain mental health with high precision. Best mental health practices propose the need to identify the early signs of risk factors such as stress. However, mental health research focuses primarily on late-stage symptom assessment via self-report data such as surveys. As a result, clinical practice for mental health mostly consists of reactive treatments. This grant seeks to advance precision mental health by developing an Artificial Intelligence (AI) -enabled platform for continuous and precise measurements of stress. This platform leverages data from a skin-like wearable device that measures cortisol, a stress hormone from sweat, and from sensing techniques based on estimating muscle stiffness changes derived from “fight or flight” stress response, by “repurposing” signals available in billions of existing mobile and computing devices. These data streams will be combined using Machine Learning (ML) algorithms for optimizing data collection, power consumption, and accuracy. Since stress and its effect on mental health deterioration are pervasive, the broader impact of this work could be enormous as well as the basis for new research on precision health in psychiatry. The overall goal of this project is to develop a multimodal sensing platform leveraging AI/ML algorithms to optimize stress prediction and hardware performance. The first step involves validating the skin-inspired wearable for lab stress measurements, followed by redesigning, and validating as a continuous in-the-wild device. It features a set of physiological sensors for collecting heart rate variability (HRV) and electrodermal activity (EDA) data, signals directly correlated with the autonomous nervous system (ANS) response. Two types of sensors for continuous measurement of cortisol level will be tested for high accuracy and selectivity. However, processing and transmitting sensor data continuously in-the-wild requires significant battery capacity. To address this issue, the second step combines the wearable with passive biomechanical sensors and AI/ML algorithms to optimize for continuous stress detection in-the-wild. Additionally, passive sensors transform computer peripheral data (e.g., mice, trackpads, smartphone screens) into parameters correlated to muscle stiffness linked to the “fight or flight” stress response. For example, the system uses inverse filtering techniques to approximate mass-spring-damper (MSD) models derived from mouse displacements or force models derived from the area under the finger on a trackpad. The system employs several AI/ML algorithms including a) compressive sensing to optimize energy efficiency, b) autoencoder models to correct for artifacts, missing data or sensor failures, c) active learning to discover optimal collection times of stress events and labels, and d) cloud computing powered data collection and processing to make predictions based on the best available data. The intellectual merits of this work include 1) a multimodal stress monitoring wearable that measures cortisol from sweat and other physiological signals, 2) biomechanical sensing algorithms that repurpose movement and touch data into muscle stiffness, and 3) AI/ML algorithms that integrate this data to optimize wearable and smartphone power usage, learn ideal sensing scenarios with high precision, improve privacy, optimize data labeling, and optimize the early prediction of stress.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

精准健康和精神病学的共同目标是精确测量相关症状并高精度维持心理健康，因此需要识别压力等危险因素的早期迹象。然而，心理健康研究主要关注晚期症状。通过自我报告数据（例如调查）进行阶段症状评估因此，心理健康的临床实践主要包括反应性治疗，该赠款旨在通过开发人工智能（AI）支持的平台来促进精确的心理健康。该平台利用数据进行精确测量。来自测量皮质醇（汗液中的一种压力荷尔蒙）的类肤可穿戴设备，以及基于估计“战斗或逃跑”压力反应所产生的肌肉僵硬变化的传感技术，通过“重新利用”数十亿现有移动和计算中可用的信号这些数据流将使用机器学习 (ML) 算法进行组合，以优化数据收集、功耗和准确性。由于压力及其对心理健康恶化的影响普遍存在，因此这项工作的更广泛影响可能是巨大的。的基础该项目的总体目标是开发一个利用人工智能/机器学习算法来优化压力预测和硬件性能的多模态传感平台，第一步是验证基于皮肤的可穿戴设备的实验室压力测量。通过重新设计和验证，它是一种连续的野外设备，具有一组生理传感器，用于收集心率变异性 (HRV) 和皮肤电活动 (EDA) 数据，这些信号与自主神经系统 (ANS) 反应直接相关。两种类型。用于连续测量皮质醇水平的传感器将进行高精度和选择性测试，但是，在野外连续处理和传输传感器数据需要大量的电池容量，为了解决这个问题，第二步将可穿戴设备与无源生物力学传感器结合起来。此外，人工智能/机器学习算法可优化野外连续压力检测，被动传感器将计算机外围设备数据（例如鼠标、触控板、智能手机屏幕）转换为与“战斗或逃跑”压力反应相关的肌肉硬度相关的参数。例如，该系统使用逆滤波技术来近似源自鼠标位移的质量-弹簧-阻尼器 (MSD) 模型或源自触控板上手指下方区域的力模型。该系统采用多种 AI/ML 算法，包括 a) 压缩传感来优化。能源效率，b) 自动编码器模型，用于纠正伪影、丢失数据或传感器故障，c) 主动学习，发现压力事件和标签的最佳收集时间，以及 d) 云计算支持的数据收集和处理，以根据最佳结果进行预测可用的这项工作的智力优点包括 1) 一种多模式压力监测可穿戴设备，可测量汗液和其他生理信号中的皮质醇，2) 将运动和触摸数据重新用于肌肉僵硬的生物力学传感算法，以及 3) 集成的 AI/ML 算法。这些数据可用于优化可穿戴设备和智能手机的功耗、高精度学习理想的传感场景、改善隐私、优化数据标签以及优化压力的早期预测。该奖项是 NSF 的法定使命，并通过使用评估被认为值得支持。基金会的智力价值和更广泛的影响审查标准。

项目成果

Optimal sets and solution paths of ReLU networks

ReLU网络的最优集和求解路径

• 发表时间: 2023-07
• 期刊: ICML'23: Proceedings of the 40th International Conference on Machine Learning
• 作者: Mishkin; Aaron
• 通讯作者: Aaron

Training Quantized Neural Networks to Global Optimality via Semidefinite Programming

通过半定规划训练量化神经网络以获得全局最优性

• 发表时间: 2021-04
• 期刊: Proceedings of Machine Learning Research
• 作者: Bartan, Burak;Pilanci, Mert
• 通讯作者: Pilanci, Mert

Convex Neural Autoregressive Models: Towards Tractable, Expressive, and Theoretically-Backed Models for Sequential Forecasting and Generation

凸神经自回归模型：面向顺序预测和生成的易于处理、富有表现力且有理论支持的模型

• DOI: 10.1109/icassp39728.2021.9413662
• 发表时间: 2021-06
• 期刊: Speech and Signal Processing (ICASSP
• 作者: Gupta, Vikul;Bartan, Burak;Ergen, Tolga;Pilanci, Mert
• 通讯作者: Pilanci, Mert

Neural Fisher Discriminant Analysis: Optimal Neural Network Embeddings in Polynomial Time

神经 Fisher 判别分析：多项式时间内的最优神经网络嵌入

• 发表时间: 2021-04
• 期刊: Proceedings of Machine Learning Research
• 作者: Bartan, Burak;Pilanci, Mert
• 通讯作者: Pilanci, Mert

Design considerations of a wearable electronic-skin for mental health and wellness: balancing bio signals and human factors

用于心理健康和保健的可穿戴电子皮肤的设计考虑因素：平衡生物信号和人为因素

• DOI: 10.1101/2021.01.20.427496
• 发表时间: 2021-01
• 期刊: bioRxiv
• 作者: Yasser Khan; Matthew L.
• 通讯作者: Matthew L.