Collaborative Research: Visual Tactile Neural Fields for Active Digital Twin Generation

合作研究：用于主动数字孪生生成的视觉触觉神经场

• 批准号: 2220868
• 负责人: Kostas Daniilidis
• 金额: $ 27.23万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2220868&HistoricalAwards=false
• 关键词: Collaborative; Research; Visual; Tactile; Neural

Robots will perform better at everyday activities when they can quickly combine their sensory data into a model of their environment, just like how humans instinctively use all their senses and knowledge to accomplish daily tasks. Robots, however, must be programmed to create these models that humans do intuitively, effortlessly, and robustly. This robotics project explores a novel algorithmic approach that combines visual and tactile sensory data with a knowledge of physics and a capability to learn that makes robot planning and reasoning more effective, efficient, and adaptable. The project includes the development and testing of research prototypes, preparation of new curriculum, and outreach to high school students and teachers and to the general public.This project introduces a new data representation, called a Visual Tactile Neural Field (VTNF), that allows robots to combine data from visual and tactile sensors to create a unified model of an object. The VTNF is designed to be used in a closed-loop manner, where a robot may use data from its physical interactions with an object to create or improve a model and may use its current understanding of a model to inform how best to interact with a physical object. Towards this end, the investigators create the mathematical techniques, computational tools, and robot hardware necessary to generate a VTNF model. The investigators also develop techniques to quantify the uncertainty about an object and use this uncertainty to learn search policies that allow robots to generate accurate models as quickly as possible. The VTNF, which allows for the easy addition of new properties about an object, provides a flexible representational foundation for other researchers and practitioners to use to enable robots to learn faster by having a more detailed understanding of both the surrounding environment and their interactions with it.This project is supported by the cross-directorate Foundational Research program in Robotics and the National Robotics Initiative, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人可以在日常活动中表现更好，当时他们可以将感觉数据迅速结合到环境模型中，就像人类本能地使用所有感官和知识来完成日常任务一样。但是，必须对机器人进行编程，以创建这些模型，这些模型是人类直观，毫不费力，稳健的。该机器人技术探索了一种新颖的算法方法，该方法将视觉和触觉感觉数据与物理知识结合在一起，并具有使机器人计划和推理更加有效，高效和适应能力的学习能力。该项目包括研究原型的开发和测试，新课程的准备以及向高中生和教师以及向公众推广。该项目引入了一种新的数据表示，称为视觉触觉神经领域（VTNF），该项目使机器人可以从视觉和触觉传感器中组合数据，以创建一个统一的对象模型。 VTNF设计为以闭环方式使用，在该方式中，机器人可以使用其与对象的物理交互中的数据来创建或改进模型，并可以使用其当前对模型的理解来告知如何最好地与物理对象进行交互。为此，研究人员创建了生成VTNF模型所需的数学技术，计算工具和机器人硬件。研究人员还开发了量化对象的不确定性的技术，并使用这种不确定性来学习搜索策略，以使机器人能够尽快生成准确的模型。 VTNF允许轻松添加有关物体的新属性，为其他研究人员和从业人员提供了灵活的代表性基础，可以通过对周围环境及其相互作用进行更详细的了解，以使机器人更快地学习及其与之的互动。该项目由该项目提供了由机器人和国民机器人和国民机器人和国民的基础研究（Interiate Interiate Insportiate Insportiate Insportiate Insportiate interiational and Interiate Insportiate Insportiate and Interiate Insportiate Insportiate and Insportiate interiational and Insportiate）的支持（科学与工程（CISE）。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛影响的评论标准来评估值得支持的。