Collaborative Research: RI: Medium: Superhuman Imitation Learning from Heterogeneous Demonstrations

合作研究：RI：媒介：异质演示中的超人模仿学习

• 批准号: 2312955
• 负责人: Brian Ziebart
• 金额: $ 79.94万
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2312955&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Medium; Superhuman

Learning from demonstrated behavior (i.e., imitation) is an effective means of knowledge transfer in animals and humans. Existing imitation learning methods for artificial intelligence (AI) systems typically assume the capabilities of the imitator match those of the demonstrator. This can lead to undesirable behavior when the imitator’s capabilities significantly exceed those of the demonstrator. This project reformulates imitation learning for AI systems that are more capable than (human) demonstrators in some aspects by seeking to make the AI system unambiguously better than human demonstrators. The project will train graduate students and undergraduates to develop artificial intelligence systems that are better aligned with safety and utility requirements in a broad range of highly impactful future applications.The project approaches its reformulated imitation learning objective using a maximum margin optimization for guiding (deep) reinforcement learning of control/decision policies. It focuses on learning from heterogeneous demonstrations and tasks that differ in quality, difficulty, and structure. Initially, multiple metrics for assessing and comparing different behaviors are assumed to be available. Later in the project, these metrics will be learned from demonstrations and supplemental annotations using deep representation learning methods. The policies produced by the approach of this project will be evaluated on a diverse set of applications: open source simulators (e.g., Atari games), manipulation and mobility tasks for robotics platforms, and cancer treatment decisions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从展示的行为（即模仿）中学习是动物和人类知识转移的有效手段。现有的模仿学习方法（AI）系统通常假设模仿者的功能与演示者的功能相匹配。当模仿者的能力大大超过演示者的能力时，这可能导致不良行为。该项目改革了对AI系统的模仿学习，这些系统在某些方面比（人类）示威者更有能力，试图使AI系统明确地比人类示威者更好地使AI系统更好。该项目将培训研究生和本科生，以开发人工智能系统，这些系统在广泛有影响力的未来应用中更好地与安全性和公用事业要求保持一致。该项目将其重新制定的模仿学习目标采用最大的指导（深度）加强控制/决策政策的强度优化。它重点是从质量，难度和结构不同的异质示威和任务中学习。最初，假定用于评估和比较不同行为的多个指标可用。在项目的后面，将使用深层表示学习方法从演示和补充注释中学到这些指标。 The policies produced by the approach of this project will be evaluated on a divers set of applications: open source simulators (e.g., Atari games), manipulation and mobility tasks for robotics platforms, and cancer treatment decisions.This award reflects NSF's statutory mission and has been deemed precious of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.