NRI: FND: Collaborative Research: DeepSoRo: High-dimensional Proprioceptive and Tactile Sensing and Modeling for Soft Grippers

NRI：FND：合作研究：DeepSoRo：软抓手的高维本体感受和触觉感知与建模

• 批准号: 2024646
• 负责人: Wenzhen Yuan
• 金额: $ 35万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-01-01 至 2023-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2024646&HistoricalAwards=false
• 关键词: NRI; FND; Collaborative; Research; DeepSoRo

This National Robotics Initiative 2.0 award supports fundamental research on fast, high-dimensional, and scalable sensing and modeling methods for soft grippers. The research will create soft grippers with significantly improved ability to handle objects in complicated environments. Soft grippers are constructed from flexible and soft materials that passively adapt to external forces, making them intrinsically safe for collaborating with humans and for handling delicate objects such as fruits and vegetables. Soft materials deform easily in response to applied forces, making them promising candidates for self-sensing. This project harnesses that promise, using embedded cameras and sophisticated algorithms to translate complex images into quantitative configuration and contact force information. Self-sensing enables soft grippers that are not limited to a preset passive response but can actively modify their operation according to their status. The active soft grippers arising from this project will find application in fields such as food industries, agriculture, assisted living for senior citizens or people with disabilities, increasing productivity and improving the quality of human life. The project follows a convergent research approach involving robotics and artificial intelligence, culminating in formal and informal learning activities to broaden the participation of underrepresented groups in engineering. This award supports the development of DeepSoRo as a new framework of integrated proprioceptive and tactile sensing using embedded cameras to provide high-dimensional sensory input, and advanced deep learning models of the gripper’s full-body kinematics and dynamics. This framework will overcome the key limitations of existing soft grippers in modeling and sensing of their own states, including the over-simplified low-resolution representation, low-speed, and difficulty in scalability and adaptability to various gripper designs. To unleash the full potential of soft grippers, several scientific boundaries must be pushed, ensuring more holistic situational awareness of those grippers to perform dexterous and safe manipulations in complex environments. This research will fill critical knowledge gaps in soft robot sensing, sensor design, and deep learning, to realize the online shape estimation and feedback control of soft grippers, especially when the grippers are in contact with external objects. This interdisciplinary research program will unfold along three directions: high dimensional shape modeling in a latent space, joint proprioceptive and tactile sensing, and sensor design and integration in hardware prototypes. Theoretical advancements will proceed alongside with experimental research toward demonstrating the potential of DeepSoRo to accurately and efficiently model and sense soft grippers in real-world settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这项国家机器人倡议2.0奖支持有关软握手的快速，高维感应和建模方法的基础研究。这项研究将创造出具有显着提高能力在复杂环境中的物体能力的软抓手。柔软的握手是由柔性和柔软的材料制成的，这些材料被动地适应外部力，使其内在安全地与人类合作以及处理诸如水果和蔬菜之类的精致物体。软材料响应于应用力而容易变形，使它们承诺候选人进行自感应。该项目利用嵌入式摄像头和复杂算法来利用该项目，将复杂的图像转化为定量配置和接触力信息。自感应使不限于预设的被动响应但可以根据其状态积极修改其操作的软握手。由该项目引起的积极柔软的抓手将在食品行业，一致派，辅助，为老年人或残疾人的辅助生活等领域找到应用，提高生产力并提高人类生活的质量。该项目遵循涉及机器人技术和人工智能的收敛研究方法，最终在正式和非正式的学习活动中，以扩大代表性不足的团体在工程中的参与。该奖项支持DeepSoro的开发，是使用嵌入式相机的综合本体感受和触觉灵敏度的新框架，以提供高维感官输入，以及对抓地力的全身运动学和动态的高级深度学习模型。该框架将克服现有软抓手在自己状态的建模和灵敏度中的关键局限性，包括过度简化的低分辨率表示，低速和对各种握把设计的可扩展性和适应性的困难。除非柔软的握手的全部潜力，否则必须突破几个科学的边界，以确保对这些抓手的更全面的情境意识，以在复杂的环境中进行灵巧和安全的操纵。这项研究将填补软机器人传感器，传感器设计和深度学习中的关键知识差距，以实现对软握手的在线形状估计和反馈控制，尤其是当抓手与外部物体接触时。该跨学科研究计划将沿三个方向展开：潜在空间中的高维形状建模，联合本体感受和触觉感官以及硬件原型中的传感器设计和集成。理论的进步将与实验研究一起进行，以证明DeepSoro在现实世界中准确有效地建模并感觉到软握手的潜力。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的审查标准来通过评估来支持的。

项目成果

Toward Zero-Shot Sim-to-Real Transfer Learning for Pneumatic Soft Robot 3D Proprioceptive Sensing

面向气动软机器人 3D 本体感知的零样本模拟到真实迁移学习

• DOI: 10.1109/icra48891.2023.10160384
• 发表时间: 2023
• 期刊: IEEE
• 作者: Yoo, Uksang;Zhao, Hanwen;Altamirano, Alvaro;Yuan, Wenzhen;Feng, Chen
• 通讯作者: Feng, Chen