NRI: FND: Collaborative Research: DeepSoRo: High-dimensional Proprioceptive and Tactile Sensing and Modeling for Soft Grippers

NRI：FND：合作研究：DeepSoRo：软抓手的高维本体感受和触觉感知与建模

• 批准号: 2024646
• 负责人: Wenzhen Yuan
• 金额: $ 35万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-01-01 至 2023-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2024646&HistoricalAwards=false
• 关键词: NRI; FND; Collaborative; Research; DeepSoRo

This National Robotics Initiative 2.0 award supports fundamental research on fast, high-dimensional, and scalable sensing and modeling methods for soft grippers. The research will create soft grippers with significantly improved ability to handle objects in complicated environments. Soft grippers are constructed from flexible and soft materials that passively adapt to external forces, making them intrinsically safe for collaborating with humans and for handling delicate objects such as fruits and vegetables. Soft materials deform easily in response to applied forces, making them promising candidates for self-sensing. This project harnesses that promise, using embedded cameras and sophisticated algorithms to translate complex images into quantitative configuration and contact force information. Self-sensing enables soft grippers that are not limited to a preset passive response but can actively modify their operation according to their status. The active soft grippers arising from this project will find application in fields such as food industries, agriculture, assisted living for senior citizens or people with disabilities, increasing productivity and improving the quality of human life. The project follows a convergent research approach involving robotics and artificial intelligence, culminating in formal and informal learning activities to broaden the participation of underrepresented groups in engineering. This award supports the development of DeepSoRo as a new framework of integrated proprioceptive and tactile sensing using embedded cameras to provide high-dimensional sensory input, and advanced deep learning models of the gripper’s full-body kinematics and dynamics. This framework will overcome the key limitations of existing soft grippers in modeling and sensing of their own states, including the over-simplified low-resolution representation, low-speed, and difficulty in scalability and adaptability to various gripper designs. To unleash the full potential of soft grippers, several scientific boundaries must be pushed, ensuring more holistic situational awareness of those grippers to perform dexterous and safe manipulations in complex environments. This research will fill critical knowledge gaps in soft robot sensing, sensor design, and deep learning, to realize the online shape estimation and feedback control of soft grippers, especially when the grippers are in contact with external objects. This interdisciplinary research program will unfold along three directions: high dimensional shape modeling in a latent space, joint proprioceptive and tactile sensing, and sensor design and integration in hardware prototypes. Theoretical advancements will proceed alongside with experimental research toward demonstrating the potential of DeepSoRo to accurately and efficiently model and sense soft grippers in real-world settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该国家机器人计划 2.0 奖项支持对软抓手的快速、高维和可扩展传感和建模方法的基础研究。该研究将显着提高软抓手在复杂环境中处理物体的能力。被动适应外力的软材料，使它们在与人类合作和处理水果和蔬菜等精致物体时本质上是安全的，软材料在施加的力的作用下很容易变形，这使得它们成为该项目的有希望的候选者。利用嵌入式摄像头和复杂的算法将复杂的图像转换为定量的配置和接触力信息，使软夹具不仅限于预设的被动响应，而且可以根据软夹具的状态主动修改其操作。该项目产生的成果将应用于食品工业、农业、老年人或残疾人的辅助生活领域，提高生产力并改善人类生活质量。该项目遵循涉及机器人和人工智能的融合研究方法，最终达到顶峰。在正式和非正式场合该奖项支持 DeepSoRo 的开发，作为一种集成本体感觉和触觉传感的新框架，使用嵌入式摄像头提供高维感觉输入和抓取器全功能的高级深度学习模型。该框架将克服现有软夹具在自身状态建模和感知方面的关键限制，包括过于简化的低分辨率表示、低速度以及可扩展性和困难性。为了充分发挥软体机械手的潜力，必须突破几个科学界限，确保这些机械手具有更全面的态势感知能力，以便在复杂的环境中进行灵巧、安全的操作。这项研究将填补软体机器人的关键知识空白。传感、传感器设计和深度学习，以实现软抓手的在线形状估计和反馈控制，特别是当抓手与外部物体接触时，这个跨学科的研究项目将沿着三个方向展开：潜在的高维形状建模。空间、关节本体感受和触觉传感、传感器设计和硬件原型集成将与实验研究一起推进，以证明 DeepSoRo 在现实环境中准确、高效地建模和感知软抓手的潜力。该奖项反映了 NSF 的法定使命和目标。通过使用基金会的智力优点和更广泛的影响审查标准进行评估，该项目被认为值得支持。

项目成果

Toward Zero-Shot Sim-to-Real Transfer Learning for Pneumatic Soft Robot 3D Proprioceptive Sensing

面向气动软机器人 3D 本体感知的零样本模拟到真实迁移学习

• DOI: 10.1109/icra48891.2023.10160384
• 发表时间: 2023-05
• 期刊: IEEE
• 作者: Yoo, Uksang;Zhao, Hanwen;Altamirano, Alvaro;Yuan, Wenzhen;Feng, Chen
• 通讯作者: Feng, Chen