CAREER: Manipulation of Novel Objects via Non-Smooth Implicit Learning

职业：通过非平滑内隐学习操纵新物体

• 批准号: 2238480
• 负责人: Michael Posa
• 金额: $ 60万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-04-01 至 2028-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2238480&HistoricalAwards=false
• 关键词: CAREER; Manipulation; Novel; Objects; via

This Faculty Early Career Development (CAREER) project will create physics-inspired learning methods for data-efficient robotic manipulation of novel objects – that is, previously unseen objects about which the robot has no prior knowledge. The result will enable future generations of robots to provide meaningful assistance throughout the daily lives of human users. To achieve this, robots must be able to quickly learn about their surroundings through physical interactions, particularly in chaotic settings beyond carefully controlled laboratory conditions. This will require robots to gain new capabilities beyond the current state of the art. This project will provide robots with the ability to determine critical characteristics of surrounding objects -- despite variations in shape, size, color, and material -- such as whether they move when touched, whether they are soft or stiff, and whether they bend or twist. A robot should be able to enter a room for the first time, briefly investigate the objects in that room, and then safely accomplish an assigned task. For example, an in-home robot might maneuver about a kitchen, encountering new food items or culinary tools, and then interact with those items to help prepare a meal. This project will advance a range of life-improving robotic applications, including in-home assistive care, dangerous search and rescue operations, or small-scale manufacturing. To train and inspire the next generation of engineers, investigators, working with educators in the Philadelphia Public School District, will develop an educational unit leveraging a robotic simulator to demonstrate concepts from high-school algebra.This project brings together concepts from non-smooth dynamics, learning, and control, to enable robots that perform dexterous manipulation of previously unseen objects, using data gathered in real time from a cluttered environment. For example, a robot may interact with a set of novel objects for at most a few seconds or minutes, then precisely perform, with human-like dexterity, complex tasks such as tool use or in-hand manipulation. The need for this project is driven first by the dependence of functional robotics on interaction between the robot and its environment, which is notoriously difficult to model, and second, by the reliance on predictive models of both model-based and sim-to-real methods for control. This project addresses the modeling of discontinuous contact-driven dynamics by gathering all sources of non-smooth behavior into a set of contact forces. An implicit loss function, which itself uses convex optimization to estimate non-smooth contact forces, can be minimized using gradient-based methods to find a set of the smooth parameters that describe the physics of robot-object interactions. To provide data-efficient learning of robot-world interaction, this project explores the following three primary research thrusts: (1) development of foundational implicit-learning frameworks with physics-inspired structure for predicting robot-world interactions, (2) dynamic manipulation of novel rigid and soft objects by unifying tactile and visual sensing with motion prediction, and (3) combining these two learning systems with control and reinforcement learning for closed-loop performance. Together, these thrusts will provide new robotic capabilities when dealing with novel objects, across a range of manipulation tasks including in-hand manipulation and object reorientation, whole-body manipulation (using limbs, torso, and other body parts) to maneuver, push, and drag heavy objects, and multi-arm manipulation of large or unwieldy objects.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该学院的早期职业发展（CAREER）项目将创建受物理启发的学习方法，用于对新物体进行数据高效的机器人操作，即机器人事先不了解的以前​​未见过的物体，其结果将使未来几代机器人能够进行操作。为了实现这一目标，机器人必须能够通过物理交互快速了解周围环境，特别是在超出严格控制的实验室条件的混乱环境中。这将需要机器人获得超越当前的新能力。该项目将达到最先进的水平。使机器人能够确定周围物体的关键特征——尽管形状、大小、颜色和材料各不相同——例如它们在触摸时是否移动、它们是软的还是硬的、以及它们是弯曲还是扭曲的。机器人应该能够首次进入一个房间，简要介绍该房间中的物体，然后安全地完成指定的任务，例如，家用机器人可能会机动地调查厨房，遇到新的食品或烹饪工具。 ，然后与这些物品互动以帮助准备饭菜。一系列改善生活的机器人应用，包括家庭辅助护理、危险搜索和救援行动或小规模制造。为了培训和激励下一代工程师、研究人员，与费城公立学区的教育工作者合作。开发一个教育单元，利用机器人模拟器来演示高中代数的概念。该项目汇集了非平滑动力学、学习和控制的概念，使机器人能够使用真实收集的数据对以前未见过的物体进行灵巧的操作时间从例如，机器人最多可以与一组新物体交互几秒钟或几分钟，然后以类似人类的灵活性精确地执行复杂的任务，例如工具使用或手动操作。该项目的驱动因素首先是功能性机器人技术对机器人与其环境之间交互的依赖，而众所周知，这种交互很难建模，其次是依赖于基于模型和模拟到真实方法的预测模型该项目致力于建模。通过将所有非平滑行为源收集到一组接触力中，可以使用基于梯度的方法来最小化隐式损失函数，该隐式损失函数本身使用凸优化来估计非平滑接触力。描述机器人与物体交互的物理过程的平滑参数集为了提供机器人与世界交互的数据高效学习，该项目探索了以下三个主要研究方向：（1）开发具有物理意义的基础隐式学习框架。受启发的结构预测机器人与世界的交互，（2）通过将触觉和视觉传感与运动预测相结合来动态操纵新颖的刚性和软物体，以及（3）将这两个学习系统与控制和强化学习相结合以实现闭环性能。推力将在处理新物体时提供新的机器人能力，涵盖一系列操纵任务，包括手动操纵和物体重新定向、全身操纵（使用四肢、躯干和其他身体部位）来操纵、推动和拖动重物对象，以及该奖项反映了 NSF 的法定使命，并且通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Im2Contact: Vision-Based Contact Localization Without Touch or Force Sensing

Im2Contact：基于视觉的接触定位，无需触摸或力感应

• 发表时间: 2023-11
• 期刊: Proceedings of Machine Learning Research
• 作者: Kim, Leon;Li, Yunshuang;Posa, Michael;Jayaraman, Dinesh
• 通讯作者: Jayaraman, Dinesh

Simultaneous Learning of Contact and Continuous Dynamics

同时学习接触和连续动力学

• 发表时间: 2023-11
• 期刊: Proceedings of Machine Learning Research
• 作者: Bianchini, Bibit;Halm, Mathew;Posa, Michael
• 通讯作者: Posa, Michael

Adaptive Contact-Implicit Model Predictive Control with Online Residual Learning

具有在线残差学习的自适应接触隐式模型预测控制

• DOI: 10.48550/arxiv.2310.09893
• 发表时间: 2023-10-15
• 期刊: ArXiv
• 作者: Wei;Alp Aydinoglu;Wanxin Jin;Michael Posa
• 通讯作者: Michael Posa