PFI:BIC: iWork, a Modular Multi-Sensing Adaptive Robot-Based Service for Vocational Assessment, Personalized Worker Training and Rehabilitation.

PFI：BIC：iWork，一种基于模块化多传感自适应机器人的服务，用于职业评估、个性化工人培训和康复。

• 批准号: 1719031
• 负责人: Fillia Makedon
• 金额: $ 99.96万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1719031&HistoricalAwards=false
• 关键词: PFI; BIC; iWork; Modular; Multi

Automation, foreign competition, and the increasing use of robots replacing human jobs, stress the need for a major shift in vocational training practices to training for intelligent manufacturing environments, so-called "Industry 4.0". In particular, vocational safety training using the latest robot and other technologies is imperative, as thousands of workers lose their job or die on the job each year due to accidents, unforeseen injuries, and lack of appropriate assessment and training. The objective of this Partnerships for Innovation: Building Innovation Capacity (PFI:BIC) project is to develop iWork, a smart robot-based vocational assessment and intervention system to assess the physical, cognitive and collaboration skills of an industry worker while he/she performs a manufacturing tasks in a simulated industry setting and collaborating with a robot to do the task. The aim is to transform traditional vocational training and rehabilitation practices to an evidence-based and personalized system that can be used to (re)train, retain, and prepare workers for robotic factories of the future. The need for personalized vocational training, rehabilitation and accurate job-matching is essential to ensuring a strong manufacturing sector, vital to America's economic development and ability to innovate. The iWork service is "smart" because it can adjust and adapt to the individual's abilities as it assesses him/her and help decide on the type of tasks needed to test and train, based on the job's complexity, difficulty or familiarity to the worker. The iWork system integrates human expert knowledge to overcome or compensate for detected worker constraints. Research has shown that robot trainers can increase motivation and sustain interest, increase compliance and learning, and provide training for specific and individual needs. The iWork system aims to assess and train both the human and the work-assistive robot, as they collaborate on a manufacturing job. The projected outcome is low-cost vocational training solutions that can have substantial economic and societal benefits to diverse economic sectors. Most importantly, if successful, projected outcomes could impact how millions of persons seeking a manufacturing job are trained, including those facing a type of learning, physical or aging disability. The system's mobile, low cost methods accelerate recognizing a worker's specific needs and improve the ability of the vocational expert to make correlations between cognitive and physical assessments, thus empowering traditional practices with user-centric targeted training methods. In addition, the project's robot-based emphasis on safety and risk assessment, can reduce liability costs and productivity setbacks faced by industry, due to manufacturing accidents. The iWork system uses computational methods in reinforcement (machine) learning, data mining, collaborative filtering and human robot interaction to collect and analyze multi-sensing worker data during a manufacturing human-robot collaboration simulation. Data collected and analyzed come from sensors, wearables, and explicit user feedback measuring worker movements, eye gazes, errors made, performance delays, human-robot interactions, physiological metrics, and others, depending on the task. The system has a closed loop architecture composed of four phases: assessment, recommendation, intervention (or adjustment), and evaluation, with a human expert in the loop. The system generates recommendations for personalized interventions to the expert, at different loop intervals. Use of the latest developments in sensing technologies, robotics and intelligent communications, assess the ability to enhance the intelligence of a robot co-worker with more human-like learning and collaboration abilities to support the human in achieving a task. The system is modular and customizable to a particular manufacturing task, domain or worker robot. Two types of robots are used, socially assistive robots that provide non-contact user assistance through feedback and physically assistive robots that provide cognitive, physical and collaboration skill training. To predict risks of injury due to inattention, age, vision, or physical and mental issues, motion analysis and kinematics experiments are conducted to determine the type of safety training needed, to assess how well a human interacts with a collaborative robot, and how best to train the robot to help the human overcome identified physical and other deficiencies in performing a given task. The project integrates three main areas of expertise, engineered service system design, where assistive robots interact with and train each other to collaborate; computing, sensing, and information technologies, where machine learning, data mining and recommender algorithms are used to identify behavioral patterns of interest, and recommend targeted interventions; and human factors and cognitive engineering that deploy methods from the team's expertise in workplace assessment, personalized psychiatric intervention, and evaluation methods of vocational satisfaction, work habits, work quality, etc., as they relate to job preparation and retention.The project has an interdisciplinary team of experts from two collaborating universities, University of Texas Arlington (UTA) and Yale University, representing several fields, including human factors, psychology, computing, and industrial organization. The project deploys two primary industry partners, SoftBank Robotics (San Francisco, CA) manufacturer of humanoid service robots, and InteraXon (Canada), producing mobile EEG devices, who provides hardware, software and know-how to enhance iWork's functionality in cognitive activity monitoring. The broader context partners include, C8Sciences (USA), Assistive Technology Resources (USA), Barrett Technologies Inc. (USA), and the Dallas Veteran Affairs Research Corp. (USA).

自动化，外国竞争以及越来越多的机器人替代人类工作的使用，强调了将职业培训实践重大转变为智能制造环境培训的必要性，即所谓的“工业4.0”。 特别是，必须使用最新机器人和其他技术的职业安全培训，因为由于事故，不可预见的伤害以及缺乏适当的评估和培训，成千上万的工人每年失业或死亡。 这项创新合作伙伴关系的目的：建立创新能力（PFI：BIC）项目是开发IWork，这是一种基于机器人的职业评估和干预系统，以评估行业工人的身体，认知和协作技能，而他/她在模拟的行业设置中执行制造任务，并与机器人合作以完成一项任务。目的是将传统的职业培训和康复实践转变为循证和个性化的系统，该系统可用于（重新）培训，保留并为工人准备未来的机器人工厂。对个性化职业培训，康复和准确的工作匹配的需求对于确保强大的制造业至关重要，这对美国的经济发展和创新能力至关重要。 IWork服务是“智能的”，因为它可以根据工作的复杂性，对工人的困难或熟悉程度来调整并适应个人的能力，并帮助确定测试和培训所需的任务类型。 IWork系统集成了人类专家知识，以克服或弥补检测到的工人限制。研究表明，机器人培训师可以增加动力并维持兴趣，增加合规性和学习，并为特定和个人需求提供培训。 IWork系统旨在评估和培训人类和工作辅助机器人，因为他们在制造业工作中进行了合作。 预计的结果是低成本的职业培训解决方案，可以为各种经济领域带来可观的经济和社会利益。最重要的是，如果成功，预计的结果可能会影响数百万寻求制造业工作的人的培训，包括那些面临一种学习，身体或衰老的残疾的人。该系统的移动，低成本方法可以加速认识工人的特定需求，并提高职业专家之间建立认知和身体评估之间相关性的能力，从而通过以用户为中心的目标培训方法赋予传统实践的能力。此外，由于制造事故，该项目基于机器人对安全和风险评估的重视可以降低行业面临的责任成本和生产力挫折。 IWork系统使用计算方法进行加固（机器）学习，数据挖掘，协作过滤和人体机器人交互，以在制造人类机器人协作模拟过程中收集和分析多感应工人数据。收集和分析的数据来自传感器，可穿戴设备和明确的用户反馈，以测量工人的动作，凝视，错误造成的错误，性能延迟，人类机器人相互作用，生理指标等，具体取决于任务。该系统具有由四个阶段组成的闭环体系结构：评估，建议，干预（或调整）以及与人类专家的循环。该系统以不同的循环间隔为专家提供个性化干预措施的建议。在感应技术，机器人技术和智能沟通中使用最新发展，评估了增强机器人同事的智慧的能力，并具有更类似人类的学习和协作能力，以支持人类完成任务。 该系统是模块化的，可根据特定的制造任务，域或工具机器人进行定制。使用了两种类型的机器人，具有社会辅助机器人，可通过反馈和提供认知，身体和协作技能培训的身体辅助机器人提供非接触用户帮助。为了预测由于不关心，年龄，视力或身体和心理问题而引起的伤害风险，进行运动分析和运动学实验，以确定所需的安全训练类型，以评估人类与协作机器人与协作机器人的互动方式，以及如何最好地培训机器人以帮助人类在执行给定的任务中确定的身体和其他缺陷。该项目集成了三个主要知识，工程服务系统设计的主要领域，辅助机器人在其中互动并互相训练以合作；计算，感应和信息技术，机器学习，数据挖掘和推荐算法用于识别感兴趣的行为模式，并推荐有针对性的干预措施；以及人为因素和认知工程，从团队在工作场所评估，个性化的精神病干预以及职业满意度，工作习惯，工作质量等的评估方法方面的专业知识中部署方法，与他们的工作准备和保留相关，该项目与得克萨斯州大学（University of Demurality of Demance of Terection of Divelliant of Divession of Sopive and Insperiation of Sopiration and Sotiveation and Sopife of Soip repartion and Progience of Sovight of Sovighation and Progiess of Sovighation and Seriveration。计算和工业组织。该项目部署了两个主要行业合作伙伴，SoftBank机器人（加利福尼亚州旧金山）人类机器人服务机器人的制造商和Interaxon（加拿大），生产移动EEG设备，他们提供了硬件，软件和专有技术，以增强IWork在认知活动监测中的功能。更广泛的背景伙伴包括C8Sciences（美国），辅助技术资源（美国），Barrett Technologies Inc.（美国）和达拉斯资深事务研究公司（美国）。

项目成果

Towards predicting task performance from EEG signals

• DOI: 10.1109/bigdata.2017.8258478
• 发表时间: 2017-12
• 期刊: 2017 IEEE International Conference on Big Data (Big Data)
• 作者: Michalis Papakostas;K. Tsiakas;Theodoros Giannakopoulos;F. Makedon

COGNITIVE ANALYSIS OF WORKING MEMORY LOAD FROM EEG, BY A DEEP RECURRENT NEURAL NETWORK

• DOI: 10.1109/icassp.2018.8462243
• 发表时间: 2018-01-01
• 期刊: 2018 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
• 作者: Kuanar, Shiba;Athitsos, Vassilis;Rao, K. R.
• 通讯作者: Rao, K. R.

Towards a Real-Time Cognitive Load Assessment System for Industrial Human-Robot Cooperation

面向工业人机合作的实时认知负荷评估系统

• DOI: 10.1109/ro-man47096.2020.9223531
• 发表时间: 2020
• 期刊: IEEE
• 作者: Rajavenkatanarayanan, Akilesh;Nambiappan, Harish Ram;Kyrarini, Maria;Makedon, Fillia
• 通讯作者: Makedon, Fillia

A Human Robot Interaction Framework for Robotic Motor Skill Learning

用于机器人运动技能学习的人机交互框架

• DOI: 10.1145/3197768.3197790
• 发表时间: 2018
• 期刊: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference on - PETRA '18
• 作者: Theofanidis, Michail;Cloud, Joe;Babu, Ashwin Ramesh;Brady, James;Makedon, Fillia
• 通讯作者: Makedon, Fillia

A Review of Extended Reality (XR) Technologies for Manufacturing Training

• DOI: 10.3390/technologies8040077
• 发表时间: 2020-12-01
• 期刊: TECHNOLOGIES
• 影响因子: 3.6
• 作者: Doolani, Sanika;Wessels, Callen;Makedon, Fillia
• 通讯作者: Makedon, Fillia