Collaborative Research: SCH: Personalized Watch-based Fall Risk Analysis and Detection with Cross Modal Learning

合作研究：SCH：通过跨模态学习进行基于手表的个性化跌倒风险分析和检测

• 批准号: 2123521
• 负责人: Yan Yan
• 金额: $ 19.35万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2123521&HistoricalAwards=false
• 关键词: Collaborative; Research; SCH; Personalized; Watch

Falls are a significant cause of morbidity and mortality in the elderly. A robust and low-cost solution for the estimation of fall risk and detection of falls will allow seniors to live independently and reduce medical costs due to falls. Wearable devices have been developed to detect “hard falls”, namely falls that cause injury. However, many falls in the elderly do not cause physical injury (“soft falls”). These occur in association with weight transfer activities such as turning and sit-stand transitions. Indeed, the ability to control the position and movements of the trunk (“core”) is essential for coordinating the movements of the limbs during weight transfer. The goal of this project is to combine real-world limb-core dynamics of an individual with data collected by accelerometer via a commodity wristwatch and a cell phone on the opposite hip to improve the detection of hard and soft falls. A personalized fall risk analysis and detection model will be created for each user via real-time learning of the limb-core dynamics using state of the art machine learning algorithm. We will also assess the perceptions and preferences of elderly patients using this technology and evaluate their attitudes towards continuous data collection and sharing of health data for improved health. The software system, the real-world gait and weight transfer movement and the associated accelerometer data will be made freely available to any institution, investigator or research student interested in the study of machine learning on health conditions as well as on fall risk and analysis. This project will train graduate and undergraduate students in technical skills (machine learning, wearable technologies and data analysis skills) as well as in people skills for working with the elderly who live in long-term care facilities. While numerous fall detection devices incorporating artificial intelligence (AI) and machine learning algorithms have been developed, this project focuses on personalizing fall risk detection. This project will explore the use of kinematic measurements of an elderly individual’s movements associated with weight transfer to enable multi-task and multi-modal machine learning algorithms to personalize fall risk detection. A small-sample-based deep learning algorithm optimized to incorporate individual kinematic characteristics using multi-task and multi-modal learning frameworks is developed. Second, the team will analyze the movement transitions captured by the Azure Kinect system in order to identify relationships between the accelerometer data and the complete skeletal frame with an emphasis on the limb-core dynamics. Specifically, our goal is to determine whether or not Generative Adversarial Networks (GANs) can be used to augment missing modality from a small amount of body motion data, smartwatch and phone acceleration data collected directly from elderly participants who are at most risk of falling, namely those living in an assisted living center. Finally, we will evaluate the perception and attitudes of the elderly participants towards the continuous use of wearable devices for fall risk analysis and detection.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

跌倒是老年人发病和死亡的一个重要原因，用于评估跌倒风险和检测跌倒的强大且低成本的解决方案将使老年人能够独立生活并减少因跌倒而产生的医疗费用。检测“硬性跌倒”，即造成伤害的跌倒。然而，许多老年人跌倒不会造成身体伤害（“软性跌倒”）。事实上，这些跌倒与体重转移活动有关，例如转身和坐站转换。控制躯干位置和运动的能力（“核心”）对于协调重量转移过程中的肢体运动至关重要，该项目的目标是将个人的真实肢体核心动态与加速度计通过商品手表和手机收集的数据结合起来。我们还将使用最先进的机器学习算法，通过实时学习肢体核心动态，为每个用户创建个性化的跌倒风险分析和检测模型。使用此评估老年患者的看法和偏好技术并评估他们对持续数据收集和共享健康数据以改善健康的态度该软件系统、现实世界的步态和体重转移运动以及相关的加速度计数据将免费提供给任何感兴趣的机构、研究人员或研究生。研究机器学习对健康状况以及跌倒风险和分析的影响 该项目将为研究生和本科生提供技术技能（机器学习、可穿戴技术和数据分析）以及与老年人一起工作的人际交往技能方面的培训。住在长期护理机构的人。已经开发了许多结合人工智能（AI）和机器学习算法的跌倒检测设备，该项目专注于个性化跌倒风险检测该项目将探索使用与体重转移相关的老年人运动的运动学测量来实现多任务。其次，该团队将开发一种基于小样本的深度学习算法，该算法可利用多任务和多模态学习框架来优化个体的运动学特征。被捕获的Azure Kinect 系统，用于识别加速度计数据和完整骨骼框架之间的关系，重点是肢体核心动力学。具体而言，我们的目标是确定是否可以使用生成对抗网络 (GAN) 来增强缺失的模态。根据直接从最有跌倒风险的老年人（即生活在辅助生活中心的老年人）收集的少量身体运动数据、智能手表和手机加速度数据，最后，我们将评估对老年人的看法和态度。持续使用可穿戴设备进行跌倒风险分析和检测。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Progressive Cross-modal Knowledge Distillation for Human Action Recognition

用于人类行为识别的渐进式跨模式知识蒸馏

• DOI: 10.1145/3503161.3548238
• 发表时间: 2022-08-17
• 期刊: Proceedings of the 30th ACM International Conference on Multimedia
• 作者: Jianyuan Ni;A. Ngu;Yan Yan
• 通讯作者: Yan Yan

Cross-Modal Knowledge Distillation For Vision-To-Sensor Action Recognition

用于视觉到传感器动作识别的跨模式知识蒸馏

• DOI: 10.1109/icassp43922.2022.9746752
• 发表时间: 2022-05
• 期刊: Speech and Signal Processing (ICASSP
• 作者: Ni, Jianyuan;Sarbajna, Raunak;Liu, Yang;Ngu, Anne H.H.;Yan, Yan
• 通讯作者: Yan, Yan

Towards Saner Deep Image Registration

迈向更加健全的深度图像配准

• DOI: 10.1109/iccv51070.2023.01145
• 发表时间: 2023-07-19
• 期刊: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: Bin Duan;Ming Zhong;Yan Yan
• 通讯作者: Yan Yan