NRI: FND: Immersive whole-body teleoperation of wheeled humanoid robots for dynamic mobil manipulation

NRI：FND：用于动态移动操纵的轮式人形机器人的沉浸式全身远程操作

• 批准号: 2024775
• 负责人: Kris Hauser
• 金额: $ 75万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2024775&HistoricalAwards=false
• 关键词: NRI; FND; Immersive; whole; body

Physically demanding labor is required of workers in construction and logistics, as well as dangerous jobs such as disaster relief, firefighting, policework, logging, and mining. Many of these workers in the US are injured, disabled, or die due to workplace incidents. Although robots have the potential to protect the well-being of these workers, they do not yet possess the capability to perform the demanding physical tasks which are inherent to these activities. A key missing aspect is the intelligence to intuitively coordinate their body to amplify the forces that they apply to objects, such as when humans naturally lean against a heavy door to open it. This project focuses on developing the technology to combine the control intelligence of humans with the physical strength and endurance of machines. In this framework, a wearable device measures the posture of the entire body of a human operator such that the remote human-like robot may reproduce the same whole-body position at the same time. The novelty of this project is to enable the human operator to simultaneously feel the forces that the robot applies to its surroundings. For instance, a firefighter commands the robot to push heavy debris by performing the motion herself, while simultaneously feeling whether the robot is exerting insufficient force and the object isn’t moving, or whether it is exerting excessive effort and losing balance. The impact of this research is the protection of the well-being of human workers by taking them away from danger, while leveraging their domain expertise and their motor intelligence to perform physically demanding labor. This research promotes the inclusion of undergraduate and underrepresented minorities in research, creates outreach activities with K-12 students, and proposes demonstrations of the system during conferences and University events.This project explores a novel bilateral teleoperation framework which enables a human operator to control a remote wheeled humanoid robot to perform Dynamic Mobile Manipulation (DMM). DMM are physical tasks which combine forceful manipulation and agile locomotion. The key hypothesis in this proposal is that bilateral teleoperation via a whole-body haptic device can achieve safe and intuitive DMM in unstructured field environments. To validate this hypothesis, three specific aims will be pursued over three years. Aim 1: Implement the hardware that enables this research, composed of (i) a prototype whole-body haptic device which reads the operator’s posture to control the remote robot, and provides rich kinesthetic, visual, and audial feedback from the robot’s interaction with its environment, and (ii) the robot named SATYRR, which combines advantages of legged systems, wheeled locomotion, and an anthropomorphic upper-body for dual-arm manipulation. Aim 2: Study whole-body bilateral teleoperation strategies for DMM. Devise a human-to-robot motion mapping for intuitive, high-bandwidth control, and implement force feedback schemes that deliver interpretable feedback to the operator even under large impacts. And Aim 3: Improve DMM safety using shared autonomy. Devise model predictive control schemes to follow the user's input as closely as possible while preventing operator inputs that would damage the robot. To evaluate the planned framework, a realistic laboratory task environment will be built with an obstacle course that includes pushing large objects, opening blocked doors, balancing under external disturbances, and carrying and throwing a payload.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

建筑和物流以及救灾、消防、警察、伐木和采矿等危险工作需要体力劳动，在美国，许多工人因工作场所事故而受伤、致残或死亡。尽管机器人有潜力保护这些工人的福祉，但它们尚不具备执行这些活动所固有的艰巨体力任务的能力，一个关键的缺失方面是直观地协调身体以放大能力的智能。他们施加的力量物体，例如当人类自然地靠在沉重的门上打开它时，该项目重点开发将人类的控制智能与机器的体力和耐力相结合的技术，在这个框架中，可穿戴设备可以测量物体的姿势。该项目的新颖之处在于使人类操作员能够同时感受到机器人施加在其上的力。例如，消防员命令机器人推动。通过自己进行运动来清除重物，同时感觉机器人是否用力不足而物体没有移动，或者是否用力过度而失去平衡。这项研究的影响是对幸福感的保护。这项研究促进了本科生和代表性不足的少数族裔参与研究，与 K-12 学生开展外展活动，并提出示范。会议和大学期间的系统该项目探索了一种新颖的双边远程操作框架，该框架使人类操作员能够控制远程轮式人形机器人来执行动态移动操纵（DMM），这是结合强力操纵和敏捷运动的物理任务。该提案的关键假设是。通过全身触觉设备进行双边远程操作可以在非结构化现场环境中实现安全直观的数字万用表。为了验证这一假设，我们将在三年内实现三个具体目标：目标 1：实现硬件。这项研究由（i）原型全身触觉设备组成，该设备读取操作员的姿势来控制远程机器人，并从机器人与其环境的交互中提供丰富的动觉、视觉和听觉反馈，以及（ii）机器人名为 SATYRR，它结合了腿式系统、轮式运动和用于双臂操纵的拟人化上半身的优点。 目标 2：研究 DMM 的全身双边远程操作策略。人机运动映射，实现直观、高带宽控制，并实施力反馈方案，即使在巨大冲击下也能向操作员提供可解释的反馈。目标 3：使用共享自主性设计模型预测控制方案来提高 DMM 的安全性。尽可能接近用户的输入，同时防止操作员的输入会损坏机器人。为了评估计划的框架，将建立一个具有障碍物的真实实验室任务环境，其中包括推动大型物体、打开堵塞的门、在外部干扰下保持平衡、并携带和投掷有效载荷。这反映了 NSF 的法定使命，并且通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

CPI: Conservativeness, Permissiveness and Intervention Metrics for Shared Control Evaluation

CPI：共享控制评估的保守性、宽容性和干预指标

• DOI: 10.1109/lra.2022.3169581
• 发表时间: 2022-07
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Zhou, Yu;Hauser, Kris
• 通讯作者: Hauser, Kris

Dynamic Locomotion Teleoperation of a Reduced Model of a Wheeled Humanoid Robot Using a Whole-Body Human-Machine Interface

使用全身人机界面的轮式人形机器人简化模型的动态运动遥控

• DOI: 10.1109/lra.2021.3138521
• 发表时间: 2022-04
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Wang, Sunyu;Ramos, Joao
• 通讯作者: Ramos, Joao

A Comparison Between Joint Space and Task Space Mappings for Dynamic Teleoperation of an Anthropomorphic Robotic Arm in Reaction Tests

反应测试中拟人机械臂动态遥操作的关节空间与任务空间映射的比较

• 发表时间: 2021-01
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Wang, Sunyu;Murphy, Kevin;Kenney, Dillan;Ramos, Joao
• 通讯作者: Ramos, Joao

The dynamic effect of mechanical losses of transmissions on the equation of motion of legged robots

传动机械损耗对腿式机器人运动方程的动态影响

• 发表时间: 2021-01
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Sim, Youngwoo;Ramos, Joao
• 通讯作者: Ramos, Joao

A Comparison Between Joint Space and Task Space Mappings for Dynamic Teleoperation of an Anthropomorphic Robotic Arm in Reaction Tests

反应测试中拟人机械臂动态遥操作的关节空间与任务空间映射的比较

• DOI: 10.1109/icra48506.2021.9561368
• 发表时间: 2021-05
• 期刊: 2021 IEEE International Conference on Robotics and Automation (ICRA
• 作者: Wang, Sunyu;Murphy, Kevin;Kenney, Dillan;Ramos, Joao
• 通讯作者: Ramos, Joao