CRII: RI: Active Learning of Preferences for Human-Aware Autonomy

CRII：RI：主动学习人类意识自主偏好

• 批准号: 1849952
• 负责人: Dorsa Sadigh
• 金额: $ 17.5万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-06-01 至 2022-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1849952&HistoricalAwards=false
• 关键词: CRII; RI; Active; Learning; Preferences

Humans' preferences play a key role in specifying how robotics systems should act, i.e., how an assistive robot arm should move, or how an autonomous car should drive. The learned human preferences are an important element in planning for interactive autonomous systems, e.g., robots collaborating with different types of human teammates, or shared autonomy with a human to efficiently teleoperate a robot arm. One hopes that learning techniques can be used to learn reward functions representing humans' preferences for robotics applications. However, a significant part of the success of learning algorithms can be attributed to the availability of large amounts of labeled data. Unfortunately, collecting and labeling data can be costly and time-consuming in robotics applications. In addition, humans are not always capable of reliably assigning a success value (reward) to a given robot action, and their demonstrations are usually suboptimal due to the difficulty of operating robots with more than a few degrees of freedom. The proposed research develops foundational techniques to address the key challenges of using learning techniques in human-robot interaction. The goal of this project is to develop efficient methods and algorithms to first better model and understand humans preferences while operating, interacting, and collaborating with robots. Furthermore, the investigator will design algorithms that plan for robots that are aware of such preferences and can initiate a safe and seamless interaction with humans. This project involves two main contributions: (1) Developing efficient and active algorithms to learn probabilistic mixture models for humans preferences about how a robot should operate based on comparisons and rankings., and (2) Developing planning algorithms for robots that leverage humans preferences to enable seamless shared autonomy and more efficient human-robot interaction. Preliminary results in the domain of autonomous driving suggest that one can learn driving preferences of humans and this approach can improve efficiency and safety of robotics systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类的偏好在指定机器人系统应如何运作方面发挥着关键作用，即辅助机器人手臂应如何移动，或者自动驾驶汽车应如何驾驶。学习到的人类偏好是规划交互式自主系统的重要元素，例如，机器人与不同类型的人类队友协作，或与人类共享自主权以有效地远程操作机器人手臂。人们希望学习技术可以用来学习代表人类对机器人应用的偏好的奖励函数。然而，学习算法成功的很大一部分可以归因于大量标记数据的可用性。不幸的是，在机器人应用中收集和标记数据可能既昂贵又耗时。此外，人类并不总是能够可靠地为给定的机器人动作分配成功值（奖励），并且由于操作具有多个自由度的机器人很困难，因此他们的演示通常不是最佳的。拟议的研究开发了基础技术，以解决在人机交互中使用学习技术的关键挑战。该项目的目标是开发有效的方法和算法，以便在与机器人操作、交互和协作时首先更好地建模和理解人类的偏好。此外，研究人员将设计算法，让机器人能够意识到这些偏好，并能够与人类进行安全、无缝的交互。该项目涉及两个主要贡献：（1）开发高效且主动的算法来学习人类偏好的概率混合模型，了解机器人应如何基于比较和排名进行操作。（2）开发机器人规划算法，利用人类偏好来实现无缝共享自主和更高效的人机交互。自动驾驶领域的初步结果表明，人们可以学习人类的驾驶偏好，这种方法可以提高机器人系统的效率和安全性。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和能力进行评估，认为值得支持。更广泛的影响审查标准。

项目成果

PLATO: Predicting Latent Affordances Through Object-Centric Play

柏拉图：通过以对象为中心的游戏来预测潜在的可供性

• DOI: 10.48550/arxiv.2203.05630
• 发表时间: 2022-03-10
• 作者: Suneel Belkhale;Dorsa Sadigh
• 通讯作者: Dorsa Sadigh

Active Learning of Reward Dynamics from Hierarchical Queries

从分层查询中主动学习奖励动态

• DOI: 10.1109/iros40897.2019.8968522
• 发表时间: 2019-11-01
• 期刊: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Ch;rayee Basu;rayee;Erdem Biyik;Zhixun He;M. Singhal;Dorsa Sadigh
• 通讯作者: Dorsa Sadigh

Learning From Imperfect Demonstrations From Agents With Varying Dynamics

从动态变化的代理的不完美演示中学习

• DOI: 10.1109/lra.2021.3068912
• 发表时间: 2021-03-10
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Zhangjie Cao;Dorsa Sadigh
• 通讯作者: Dorsa Sadigh

Learning Human Objectives from Sequences of Physical Corrections

从一系列物理矫正中学习人类目标

• 发表时间: 2021-05
• 期刊: 2021 IEEE International Conference on Robotics and Automation
• 作者: Li, Mengxi;Canberk, Alper;Losey, Dylan;Sadigh, Dorsa
• 通讯作者: Sadigh, Dorsa

The Green Choice: Learning and Influencing Human Decisions on Shared Roads

绿色选择：学习并影响人类在共享道路上的决策

• DOI: 10.1109/cdc40024.2019.9030169
• 发表时间: 2019-12
• 期刊: IEEE Conference on Decision and Control (CDC
• 作者: Biyik, Erdem;Lazar, Daniel A.;Sadigh, Dorsa;Pedarsani, Ramtin
• 通讯作者: Pedarsani, Ramtin