A Framework for Semi-Autonomous In-Hand Telemanipulation

半自主手持遥控框架

• 批准号: 2114464
• 负责人: Xiaoli Zhang
• 金额: $ 47.34万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2114464&HistoricalAwards=false
• 关键词: Framework; Semi; Autonomous; Hand; Telemanipulation

This award supports research to develop and test a control framework for stable in-hand telemanipulation, using a physic-informed machine learning approach that can adapt the telerobotic controller to physiological limitations of individual users. In daily life, in-hand manipulation is needed to complete seemingly simple tasks such as adjusting the orientation of a power adaptor for plugging in to an outlet, as well as to complete complex skilled tasks such as manipulating a scalpel during a surgical procedure. The long-term objective of the line of research initiated with this project is to enable dexterous in-hand manipulation in tele-operation scenarios in which the robot can understand how to track real-time changes in human finger motions and use that information to actively ensure the stability of an in-hand object while it is being manipulated. This project promotes the progress of science and advances the national health, prosperity, and welfare by developing and testing novel methods implementing stable shared human and machine control of hand-held objects in tele-operation scenarios, such as tele-surgery, healthcare assistive robotics, and remote search and rescue. The project will also support outreach activities through an existing K-12 program at the Colorado School of Mines, at a local community college, and at STEM camp for girls.This project seeks to solve challenges associated with in-hand telemanipulation through the development and testing of a control framework that utilizes physic-informed hierarchical machine learning approaches to adapt the telerobot to the physiological limitations of the individual users. There are three research objectives: 1) to use physics-informed metrics to guide the robot control policy to generate stable grasp configurations; 2) to optimize interpretation of signals derived from the human hand motion tracking system using a hierarchical learning model; and 3) to personalize shared control of the manipulation task between the human and the robot through active machine learning guided by the user's corrective adjustments in real-time. The project team will use a commercially available robotic hand system with the semi-autonomous framework to conduct human-subject-involved evaluation. Subjects will perform tele-manipulation tasks of increasing complexity, ranging from the relative simplicity of a jar opening task to a more complex case requiring in-hand changes of tool position and orientation. The work promises to advance the science and engineering of human-robot cooperation through a novel semi-autonomous control framework that can support complex in-hand object manipulation for teleoperation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项支持研究和测试用于稳定的远程接触的控制框架，使用物理信息的机器学习方法，该方法可以使远程动物控制器适应个人用户的生理限制。在日常生活中，需要手持操作来完成看似简单的任务，例如调整电源适配器的方向以插入出口，并完成复杂的熟练任务，例如在手术过程中操纵手术刀。通过该项目启动的研究线的长期目标是在电视操作场景中实现灵巧的操纵操纵，使机器人可以理解如何跟踪人手指运动的实时变化，并使用该信息来积极确保操纵时手工对象的稳定性。该项目通过开发和测试新的方法来促进科学的进步，并在远程手术，医疗保健辅助机器人和远程搜索和远程搜索中实施稳定的人类和机器控制的新方法，从而促进了国家健康，繁荣和福利的进步。 该项目还将通过科罗拉多州矿业，当地社区学院的现有K-12计划以及在STEM营地为女孩提供支持。该项目旨在通过开发和测试控制和测试的控制框架来解决与远程远程消除有关的挑战，该框架利用了物理形成的高级机器学习方法，以适应远距离的物理学限制，以适应物理学限制。有三个研究目标：1）使用物理信息指标来指导机器人控制策略生成稳定的掌握配置； 2）使用层次学习模型优化从人体运动跟踪系统得出的信号的解释； 3）通过主动机器学习在用户实时的纠正措施指导下，通过主动机器学习来个性化对人与机器人之间的操纵任务的共同控制。该项目团队将使用具有半自治框架的市售机器人手系统进行人体受试者涉及的评估。受试者将执行提高复杂性的远程操作任务，范围从jar打开任务的相对简单性到更复杂的案例，需要手动更改工具位置和方向。这项工作有望通过一个新型的半自主控制框架来推进人类机器人合作的科学和工程，该框架可以支持复杂的手机操纵对遥控的操纵。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的审查标准来通过评估来支持的。

项目成果

Robust Motion Mapping Between Human and Humanoids Using CycleAutoencoder

• DOI: 10.1109/robio54168.2021.9739345
• 发表时间: 2021-12
• 期刊: 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO)
• 作者: Matthew Stanley;Lingfeng Tao;Xiaoli Zhang

WE-Filter: Adaptive Acceptance Criteria for Filter-based Shared Autonomy

WE-Filter：基于过滤器的共享自治的自适应验收标准

• DOI: 10.1109/icra48891.2023.10161228
• 发表时间: 2023
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Bowman, Michael;Zhang, Xiaoli
• 通讯作者: Zhang, Xiaoli

Learn and Transfer Knowledge of Preferred Assistance Strategies in Semi-Autonomous Telemanipulation

• DOI: 10.1007/s10846-022-01596-2
• 发表时间: 2020-03
• 期刊: Journal of Intelligent & Robotic Systems
• 影响因子: 3.3
• 作者: Lingfeng Tao;Michael Bowman;Xu Zhou;Jiucai Zhang;Xiaoli Zhang

Dimension-Specific Shared Autonomy for Handling Disagreement in Telemanipulation

• DOI: 10.1109/lra.2023.3239313
• 发表时间: 2023-03
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Michael Bowman;Xiaoli Zhang

A Multi-Agent Approach for Adaptive Finger Cooperation in Learning-based In-Hand Manipulation

• DOI: 10.1109/icra48891.2023.10160909
• 发表时间: 2022-10
• 期刊: 2023 IEEE International Conference on Robotics and Automation (ICRA)
• 作者: Lingfeng Tao;Jiucai Zhang;Michael Bowman;Xiaoli Zhang