Collaborative Research: DARE: A Personalized Assistive Robotic System that assesses Cognitive Fatigue in Persons with Paralysis

合作研究：DARE：一种评估瘫痪者认知疲劳的个性化辅助机器人系统

• 批准号: 2226164
• 负责人: Fillia Makedon
• 金额: $ 21.83万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2226164&HistoricalAwards=false
• 关键词: Collaborative; Research; DARE; Personalized; Assistive

With the advancements in robotics and artificial intelligence, assistive robotic systems have the potential to provide support and care to people with Spinal Cord Injury (SCI). As robots become more widespread, like today’s mobile phones, assistive robots can play a significant role in assisting persons with disabilities at home, improving independence and everyday quality of life. For example, a robot may assist an individual with motor impairments to perform a task, such as preparing lunch. Current research focuses on ensuring safe human-robot cooperation in industrial environments, which will not threaten or harm the physical health of the human teammate. However, there is limited research on understanding the cognitive or mental state of a human who cooperates with an assistive robot daily for performing activities of daily living. This is important for developing effective, intuitive, and personalized human-robot cooperation. The objective of this project is to design and develop an end-to-end personalized assistive robotic system, called iRCSA (Intelligent Robotic Cooperation for Safe Assistance), to recognize, assess, and respond to a human’s cognitive fatigue during human-robot cooperation. The focus of the system is on human-robot cooperative tasks where a human with SCI and a robot cooperate during daily tasks (e.g., cooking). Students who have experienced SCI will be involved in every stage of the project, to ensure the acceptability and usability of the proposed system. In addition to the significant impact of this research on the improvement of life independence for persons with disabilities, the project includes the development of new university courses for assistive technologies and summer school programs for K-12 students, so that students gain knowledge on robotics and assistive technologies for their prospective studies in Science, Technology, Engineering, and Math (STEM). This project will develop an end-to-end framework for online cognitive fatigue assessment as part of the proposed Human-Robot Cooperation (HRC) system, which constitutes a breakthrough in the ability to effectively integrate the human component in the adaptation (personalization) of assistive robots and address key HRC challenges through these three thrusts: (a) The development of a novel human-technology system for online cognitive fatigue assessment using multimodal data; (b) the development of an adaptive robotic system for personalized interaction based on cognitive fatigue assessment; and (c) an experimental testbed of HRC scenarios to enable HRC, machine/deep learning, and robotic systems computational advances. In addition to detecting a user’s cognitive fatigue, the HRC system assesses the level of cognitive fatigue severity so that new methodologies can be designed to provide personalized robotic interactions based on the user’s state of cognitive fatigue. A participatory action research approach will be followed involving students with motor impairments throughout the project. Evaluation of the system will identify system benefits and weaknesses to improve HRC for persons with paralysis, as well as assess system usability, reliability, and user experience. Beyond the assistive robotic applications, the project will advance research in multimodal machine learning for intelligent online assessment and control in several fields where teaming between robots and humans is required.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着机器人技术和人工智能的进步，辅助机器人系统有可能为脊髓损伤 (SCI) 患者提供支持和护理。随着机器人变得越来越普遍，就像今天的手机一样，辅助机器人可以在协助方面发挥重要作用。例如，机器人可以帮助有运动障碍的人执行任务，例如准备午餐，目前的研究重点是确保工业环境中的安全人机合作。哪个不会威胁或损害人类队友的身体健康。然而，对于了解每天与辅助机器人合作进行日常生活活动的人类的认知或心理状态的研究有限，这对于有效发展、该项目的目标是设计和开发一种端到端的个性化辅助机器人系统，称为 iRCSA（安全援助智能机器人合作），以识别、评估和响应。人类的该系统的重点是人机合作任务，其中具有 SCI 的人类和机器人在日常任务（例如烹饪）中进行合作，经历过 SCI 的学生将参与其中的每个阶段。该项目旨在确保拟议系统的可接受性和可用性。除了这项研究对改善残疾人生活独立性的重大影响外，该项目还包括开发新的辅助技术大学课程和暑期学校项目。对于 K-12 学生，所以该项目将开发一个端到端的在线认知疲劳评估框架，作为拟议的人类机器人的一部分。合作（HRC）系统，它构成了在辅助机器人的适应（个性化）中有效整合人类成分的能力的突破，并通过以下三个推动力解决关键的HRC挑战：（a）开发新型人类技术系统用于在线认知疲劳评估多模态数据；(b) 基于认知疲劳评估的个性化交互的自适应机器人系统；(c) HRC 场景的实验测试台，以实现 HRC、机器/深度学习和机器人系统计算进步。 HRC 系统检测用户的认知疲劳，评估认知疲劳的严重程度，以便设计新的方法，根据用户的认知疲劳状态提供个性化的机器人交互，将全程跟踪有运动障碍的学生。该项目的评估将确定系统的优点和缺点，以改善瘫痪者的 HRC，并评估系统的可用性、可靠性和用户体验。除了辅助机器人应用之外，该项目还将推进多模式机器学习的研究。在需要机器人和人类合作的多个领域进行智能在线评估和控制。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Assessment of Cognitive Fatigue from Gait Cycle Analysis

通过步态周期分析评估认知疲劳

• DOI: 10.3390/technologies11010018
• 发表时间: 2023-02
• 期刊: Technologies
• 影响因子: 3.6
• 作者: Pavel, Hamza Reza;Karim, Enamul;Jaiswal, Ashish;Acharya, Sneh;Nale, Gaurav;Theofanidis, Michail;Makedon, Fillia
• 通讯作者: Makedon, Fillia

An EEG-based Cognitive Fatigue Detection System

基于脑电图的认知疲劳检测系统

• DOI: 10.1145/3594806.3594848
• 发表时间: 2023-07-05
• 期刊: Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments
• 作者: Enamul Karim;Hamza Reza Pavel;Ashish Jaiswal;Mohammad Zaki Zadeh;Michail Theofanidis;G. Wylie;F. Makedon
• 通讯作者: F. Makedon

A Smart Sensor Suit (SSS) to Assess Cognitive and Physical Fatigue with Machine Learning

通过机器学习评估认知和身体疲劳的智能传感器套装 (SSS)

• 发表时间: 2023-07
• 期刊: HCII 2023
• 作者: Jaiswal, Ashish;Zadeh, Mohammad Zaki;Hebri, Aref;Babu, Ashwin Ramesh;Makedon, Fillia.
• 通讯作者: Makedon, Fillia.