Collaborative Research: DARE: A Personalized Assistive Robotic System that assesses Cognitive Fatigue in Persons with Paralysis

合作研究：DARE：一种评估瘫痪者认知疲劳的个性化辅助机器人系统

• 批准号: 2226164
• 负责人: Fillia Makedon
• 金额: $ 21.83万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2026-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2226164&HistoricalAwards=false
• 关键词: Collaborative; Research; DARE; Personalized; Assistive

With the advancements in robotics and artificial intelligence, assistive robotic systems have the potential to provide support and care to people with Spinal Cord Injury (SCI). As robots become more widespread, like today’s mobile phones, assistive robots can play a significant role in assisting persons with disabilities at home, improving independence and everyday quality of life. For example, a robot may assist an individual with motor impairments to perform a task, such as preparing lunch. Current research focuses on ensuring safe human-robot cooperation in industrial environments, which will not threaten or harm the physical health of the human teammate. However, there is limited research on understanding the cognitive or mental state of a human who cooperates with an assistive robot daily for performing activities of daily living. This is important for developing effective, intuitive, and personalized human-robot cooperation. The objective of this project is to design and develop an end-to-end personalized assistive robotic system, called iRCSA (Intelligent Robotic Cooperation for Safe Assistance), to recognize, assess, and respond to a human’s cognitive fatigue during human-robot cooperation. The focus of the system is on human-robot cooperative tasks where a human with SCI and a robot cooperate during daily tasks (e.g., cooking). Students who have experienced SCI will be involved in every stage of the project, to ensure the acceptability and usability of the proposed system. In addition to the significant impact of this research on the improvement of life independence for persons with disabilities, the project includes the development of new university courses for assistive technologies and summer school programs for K-12 students, so that students gain knowledge on robotics and assistive technologies for their prospective studies in Science, Technology, Engineering, and Math (STEM). This project will develop an end-to-end framework for online cognitive fatigue assessment as part of the proposed Human-Robot Cooperation (HRC) system, which constitutes a breakthrough in the ability to effectively integrate the human component in the adaptation (personalization) of assistive robots and address key HRC challenges through these three thrusts: (a) The development of a novel human-technology system for online cognitive fatigue assessment using multimodal data; (b) the development of an adaptive robotic system for personalized interaction based on cognitive fatigue assessment; and (c) an experimental testbed of HRC scenarios to enable HRC, machine/deep learning, and robotic systems computational advances. In addition to detecting a user’s cognitive fatigue, the HRC system assesses the level of cognitive fatigue severity so that new methodologies can be designed to provide personalized robotic interactions based on the user’s state of cognitive fatigue. A participatory action research approach will be followed involving students with motor impairments throughout the project. Evaluation of the system will identify system benefits and weaknesses to improve HRC for persons with paralysis, as well as assess system usability, reliability, and user experience. Beyond the assistive robotic applications, the project will advance research in multimodal machine learning for intelligent online assessment and control in several fields where teaming between robots and humans is required.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着机器人技术和人工智能的进步，辅助机器人系统有可能为脊髓损伤（SCI）的人提供支持和护理。随着机器人的宽度越来越大，像当今的手机一样，辅助机器人可以在协助残疾人在家，改善独立性和每天生活质量方面发挥重要作用。例如，机器人可以帮助个人有运动障碍来执行任务，例如准备午餐。当前的研究重点是确保在工业环境中安全的人类机器人合作，这不会威胁或损害人类队友的身体健康。但是，关于了解人类的认知或精神状态的研究有限，他们每天与辅助机器人合作进行日常生活活动。这对于发展有效，直观和个性化的人类机器人合作很重要。该项目的目的是设计和开发一种名为IRCSA的端到端个性化辅助机器人系统（智能机器人合作以获得安全援助），以识别，评估和应对人类机器人合作期间人类的认知疲劳。该系统的重点是人机合作任务，在日常任务（例如烹饪）中，具有SCI和机器人合作的人类合作。经历过SCI的学生将参与项目的每个阶段，以确保拟议系统的可接受性和可用性。除了这项研究对残疾人的生活独立性改善的重大影响外，该项目还包括开发针对K-12学生的辅助技术和暑期学校课程的新大学课程，以便学生获得有关机器人技术和辅助技术的知识，以了解其在科学，技术，工程，工程和数学（STEM）方面的前瞻性研究。该项目将开发一个作为拟议的人类机器人合作（HRC）系统的一部分的在线认知疲劳评估的端到端框架，该系统构成了有效地整合人类组成部分（个性化）在辅助机器人中的人类组成部分的突破，并使用这些三个推力的人类效力来解决辅助机器人的关键挑战：（A） 数据; （b）基于认知疲劳评估的个性化相互作用的自适应机器人系统的开发； （c）对HRC方案的实验测试，以实现HRC，机器/深度学习和机器人系统计算进步。除了检测用户的认知疲劳外，HRC系统还评估了认知疲劳严重程度的水平，因此可以设计新方法以根据用户的认知疲劳状态提供个性化的机器人互动。将采用参与行动研究方法，使整个项目中有运动障碍的学生参与。对系统的评估将确定系统利益和弱点，以改善瘫痪者的HRC，以及评估系统的可用性，可靠性和用户体验。除了辅助机器人应用外，该项目还将在需要机器人和人类之间的几个领域进行多模式机器学习的研究，以进行智能在线评估和控制。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的审查标准来通过评估来获得的支持。

项目成果

Assessment of Cognitive Fatigue from Gait Cycle Analysis

通过步态周期分析评估认知疲劳

• DOI: 10.3390/technologies11010018
• 发表时间: 2023
• 期刊: Technologies
• 影响因子: 3.6
• 作者: Pavel, Hamza Reza;Karim, Enamul;Jaiswal, Ashish;Acharya, Sneh;Nale, Gaurav;Theofanidis, Michail;Makedon, Fillia
• 通讯作者: Makedon, Fillia

An EEG-based Cognitive Fatigue Detection System

• DOI: 10.1145/3594806.3594848
• 发表时间: 2023-07
• 期刊: Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments
• 作者: Enamul Karim;Hamza Reza Pavel;Ashish Jaiswal;Mohammad Zaki Zadeh;Michail Theofanidis;G. Wylie;F. Makedon

A Smart Sensor Suit (SSS) to Assess Cognitive and Physical Fatigue with Machine Learning

通过机器学习评估认知和身体疲劳的智能传感器套装 (SSS)

• 发表时间: 2023
• 期刊: HCII 2023
• 作者: Jaiswal, Ashish;Zadeh, Mohammad Zaki;Hebri, Aref;Babu, Ashwin Ramesh;Makedon, Fillia.
• 通讯作者: Makedon, Fillia.