Collaborative Research: CPS: Medium: Robotic Perception and Manipulation via Full-Spectral Wireless Sensing

合作研究：CPS：媒介：通过全光谱无线传感进行机器人感知和操纵

• 批准号: 2313234
• 负责人: Fadel Adib
• 金额: $ 60万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-06-01 至 2026-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313234&HistoricalAwards=false
• 关键词: Collaborative; Research; CPS; Medium; Robotic

Robotic manipulation and automation systems have received a lot of attention in the past few years and have demonstrated promising performance in various applications spanning smart manufacturing, remote surgery, and home automation. These advances have been partly due to advanced perception capabilities (using vision and haptics) and new learning models and algorithms for manipulation and control. However, state-of-the-art cyber-physical systems remain limited in their sensing and perception to a direct line of sight and direct contact with the objects they need to perceive. The goal of this project is to design, build, and evaluate a cyber-physical system that can sense, perceive, learn, and manipulate far beyond what is feasible using existing systems. To do so, the research will explore the terahertz band, which offers a new sensing dimension by inferring the inherent material properties of objects via wireless terahertz signals and without direct contact. This project will also explore radio-frequency signals that can traverse occlusions. Building on these emerging sensing modalities, the core of the project focuses on developing full-spectrum perception, control, learning, and manipulation tasks. The success of this project will result in CPS system architectures with unprecedented capabilities, enabling fundamentally new opportunities to make robotic manipulation more efficient and allowing robots to perform new complex tasks that have not been possible before. The project will enable robotic perception via full-spectral wireless sensing in order to unlock unprecedented robotic manipulation capabilities. This research involves learning synergies between sensing and control- whereby sensing is used for control and vice-versa - to optimize the end-to-end cyber-physical tasks. In particular, this research includes three inter-connected thrusts: (i) It will enable a new sensing modality that exploits high-resolution terahertz frequencies for robotic imaging and inference; (ii) It aims to build a new learning platform for full spectrum (mmWave, THz, and vision) perception to enable beyond-vision perception and reasoning in non-line-of-sight and cluttered environments, where optical systems lack in performance; and (iii) It presents a platform to learn the synergies between sensing and control to further co-optimize the end-to-end robotic manipulations tasks. These capabilities can open up entirely new realms of possibility to industrial robotics as well as assistive, warehousing, and smart home robotic. The research will be evaluated through extensive experimentation, prototype design, and system implementation. The results will be disseminated through close collaboration with industry and publications in top research venues.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人操纵和自动化系统在过去几年中受到了广泛关注，并在智能制造、远程手术和家庭自动化等各种应用中表现出了良好的性能。这些进步部分归功于先进的感知能力（使用视觉和触觉）以及用于操纵和控制的新学习模型和算法。然而，最先进的网络物理系统的感知和感知仍然仅限于直接视线以及与需要感知的物体的直接接触。该项目的目标是设计、构建和评估一个网络物理系统，该系统可以感知、感知、学习和操作，远远超出现有系统的可行性。为此，该研究将探索太赫兹频段，该频段通过无线太赫兹信号且无需直接接触来推断物体固有的材料特性，从而提供了新的传感维度。 该项目还将探索可以穿越遮挡物的射频信号。基于这些新兴的传感模式，该项目的核心重点是开发全谱感知、控制、学习和操纵任务。该项目的成功将带来具有前所未有能力的 CPS 系统架构，从根本上带来新的机会，使机器人操作更加高效，并允许机器人执行以前不可能的新复杂任务。该项目将通过全光谱无线传感实现机器人感知，以释放前所未有的机器人操纵能力。这项研究涉及学习传感和控制之间的协同作用——传感用于控制，反之亦然——以优化端到端网络物理任务。特别是，这项研究包括三个相互关联的主旨：（i）它将实现一种新的传感模式，利用高分辨率太赫兹频率进行机器人成像和推理； (ii) 旨在建立一个新的全光谱（毫米波、太赫兹和视觉）感知学习平台，以在光学系统性能不足的非视距和杂乱环境中实现超视觉感知和推理； (iii)它提供了一个学习传感和控制之间协同作用的平台，以进一步共同优化端到端机器人操作任务。这些功能可以为工业机器人以及辅助、仓储和智能家居机器人开辟全新的可能性领域。该研究将通过广泛的实验、原型设计和系统实施进行评估。结果将通过与业界和顶级研究场所的出版物密切合作来传播。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。