RI: CAREER: Task-Oriented Model Identification for Robust Robotic Manipulation

RI：职业：鲁棒机器人操作的面向任务的模型识别

• 批准号: 1846043
• 负责人: Abdeslam Boularias
• 金额: $ 53.59万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1846043&HistoricalAwards=false
• 关键词: RI; CAREER; Task; Oriented; Model

Robots typically rely on models of their mechanical components and of the objects in their surroundings to perform their tasks reliably. The models describe the shapes and the mechanical properties of objects. The models are used to simulate different actions that a robot can perform, and the actions with the best forecasted outcomes are selected for execution in the real environment. In practice, the forecasted outcomes are often different from the real outcomes due to the inaccuracies of the models, this difference is what is called the reality gap. Manually-designed models are inherently inaccurate. While this problem is less pronounced in industrial robots that typically operate in closed, structured and controlled environments, it severely limits the deployment of robots to open environments where they constantly encounter novel objects with unknown or uncertain models. For example, an assistant robot in a repair shop needs to manipulate various tools and operate on new objects everyday. The goal of this project is to develop automated and data-driven object modeling methods that will allow robots to build geometric and mechanical models of objects on the fly while manipulating them cautiously. Anticipated improvements have the potential for impact in several application areas, such as job shops that require high flexibility in product engineering, household robotics, and debris removal in rescue operations. This project fosters these potentials by creating a new course and textbook in robot learning, and releasing general purpose object modeling tools, while organizing museum exhibitions that will expose automated object modeling and manipulation techniques to a wider audience. Additionally, the project seeks to involve undergraduates in research activities at Rutgers, The State University of New Jersey, which serves a diverse student population.The approach pursued in this project is to automatically generate and gradually fine-tune mechanical models of objects by searching for models that minimize the gaps between simulation and reality. Specifically, the goal here is not to identify the most accurate model of an object, but rather to infer models that are sufficiently accurate to perform a given manipulation task. Therefore, the automated modeling process is strongly guided by the given manipulation task, unnecessary computational modeling efforts are thus avoided. The main technical objectives of this project are to: 1) Provide theoretical guarantees on the performance of control techniques using imperfect models inferred from data. 2) Develop black-box Bayesian optimization tools for inferring models of objects from limited vision and interaction data. 3) Develop white-box model identification tools using differentiable 3D renderers and physics engines. 4) Demonstrate the developed methods on a diverse range of tasks related to manipulating unknown objects in cluttered environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人通常依靠其机械组件的模型和周围的物体的模型来可靠地执行其任务。这些模型描述了对象的形状和机械性能。这些模型用于模拟机器人可以执行的不同操作，并选择了最佳预测结果的动作以在真实环境中执行。实际上，预测的结果通常与由于模型不准确而导致的实际结果不同，这种差异就是所谓的现实差距。手动设计的模型本质上是不准确的。尽管在通常在封闭，结构化和控制的环境中运行的工业机器人中，此问题并不明显，但它严重限制了机器人的部署到开放环境中，它们不断遇到具有未知或不确定模型的新颖对象。例如，维修店中的助理机器人需要操纵各种工具并每天在新物体上操作。该项目的目的是开发自动化和数据驱动的对象建模方法，这些方法将使机器人可以随时构建对象的几何和机械模型，同时谨慎地操纵它们。预期的改进有可能在几个应用领域产生影响，例如需要在救援行动中清除产品工程，家用机器人技术和清除碎屑的车间。该项目通过在机器人学习中创建新课程和教科书来促进这些潜力，并发布通用对象建模工具，同时组织博物馆展览会，以使自动化的对象建模和操纵技术暴露给更广泛的受众。此外，该项目旨在让新泽西州立大学罗格斯大学的研究活动参与本科生，该大学服务于多样化的学生人口。该项目中所采用的方法是通过搜索最小化模型和现实之间的模型来自动生成并逐渐生成并逐渐生成对象的微调机械模型。具体而言，这里的目标不是确定对象的最准确模型，而是要推断出足够准确的模型，可以执行给定的操作任务。因此，自动化建模过程在给定的操纵任务的强烈指导下，因此避免了不必要的计算建模工作。该项目的主要技术目标是：1）使用从数据推断出的不完美模型对控制技术的性能提供理论保证。 2）开发黑盒贝叶斯优化工具，用于从有限的视觉和交互数据中推断对象模型。 3）使用可区分的3D渲染器和物理引擎开发白色框模型标识工具。 4）展示了与操纵杂乱环境中未知物体有关的各种任务的开发方法。该奖项反映了NSF的法定任务，并且使用基金会的知识分子优点和更广泛的影响审查标准，被认为值得通过评估来支持。

项目成果

Interleaving Monte Carlo Tree Search and Self-Supervised Learning for Object Retrieval in Clutter

• DOI: 10.1109/icra46639.2022.9812132
• 发表时间: 2022-02
• 期刊: 2022 International Conference on Robotics and Automation (ICRA)
• 作者: Baichuan Huang;Teng Guo;Abdeslam Boularias;Jingjin Yu

Toward Fully Automated Metal Recycling using Computer Vision and Non-Prehensile Manipulation

使用计算机视觉和非预握操作实现全自动金属回收

• 发表时间: 2021
• 期刊: Proceedings of the 17th IEEE International Conference on Automation Science and Engineering
• 作者: Han, Shuai;Huang, Baichuan;Song, Changkyu;Feng, Si Wei;Xu, Ming;Boularias, Abdeslam;Yu, Jingjin
• 通讯作者: Yu, Jingjin

Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations

从视觉演示中学习顺序操作任务的感觉运动原语

• DOI: 10.1109/icra46639.2022.9811703
• 发表时间: 2022
• 期刊: International Conference on Robotics and Automation (ICRA
• 作者: Liang, Junchi;Wen, Bowen;Bekris, Kostas;Boularias, Abdeslam
• 通讯作者: Boularias, Abdeslam

A Self-supervised Learning System for Object Detection in Videos Using Random Walks on Graphs

• DOI: 10.1109/icra48506.2021.9561271
• 发表时间: 2020-11
• 期刊: 2021 IEEE International Conference on Robotics and Automation (ICRA)
• 作者: Juntao Tan;Changkyu Song;Abdeslam Boularias

Identifying Mechanical Models through Differentiable Simulations

通过可微分模拟识别机械模型

• 发表时间: 2020
• 期刊: Proceedings of Machine Learning Research
• 作者: Song, Changkyu;Boularias, Abdeslam
• 通讯作者: Boularias, Abdeslam