NRI: Collaborative Research: A Framework for Hierarchical, Probabilistic Planning and Learning

NRI：协作研究：分层、概率规划和学习的框架

• 批准号: 1637614
• 负责人: Stefanie Tellex
• 金额: $ 54.27万
• 依托单位: Brown University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1637614&HistoricalAwards=false
• 关键词: NRI; Collaborative; Research; Framework; Hierarchical

This project is an effort to create a unified framework for solving very large problems with uncertain states and actions, such as manipulator robots acting in real-world environments. The results may have especially great promise for assistive technologies, including autonomous robots that can be used by elderly and disabled populations to aid them in their daily activities. The proposed integrated framework will represent, apply, and learn hierarchical domain knowledge, and will include the ability to transfer knowledge from simpler problems to more complex ones. The research will enable autonomous agents to develop a structured representation of complex domains based on experience. The agents will use learned representations to interpret natural language commands for both low-level and high-level requests. The technical focus is enabling tractable planning in large, uncertain domains by generating and leveraging probabilistic domain knowledge at multiple levels of abstraction. Agents will autonomously create layered representations in which the layers build on one another to produce complex behaviors. Agents will learn to perform useful behaviors, such as navigating using low-level sensor feedback or assembling complex objects such as a bridge or a table. The key technical contributions will be methods for (1) planning in large state/action spaces using the abstract object-oriented Markov decision process (AMDP) model, a new formalism for representing probabilistic domain knowledge at multiple levels of abstraction; (2) learning hierarchical task knowledge in the form of AMDPs; and (3) interpreting natural language commands at multiple levels of abstraction by mapping to the learned hierarchical structure. The formalism will be demonstrated and validated in several domains, including a simulated "cleanup" toy domain, challenging and complex video games, and a robot manipulation task.

该项目致力于创建一个统一的框架，用于解决具有不确定状态和动作的非常大的问题，例如在现实环境中运行的机械手机器人。 研究结果可能对辅助技术有特别大的前景，包括可供老年人和残疾人使用的自主机器人来帮助他们进行日常活动。 所提出的集成框架将表示、应用和学习分层领域知识，并将包括将知识从更简单的问题转移到更复杂的问题的能力。该研究将使自主代理能够根据经验开发复杂领域的结构化表示。代理将使用学习到的表示来解释低级和高级请求的自然语言命令。 技术重点是通过在多个抽象级别生成和利用概率领域知识，在大型、不确定的领域中实现易于处理的规划。代理将自主创建分层表示，其中各层相互构建以产生复杂的行为。智能体将学习执行有用的行为，例如使用低级传感器反馈进行导航或组装复杂的物体（例如桥梁或桌子）。 关键的技术贡献将是以下方法：（1）使用抽象的面向对象的马尔可夫决策过程（AMDP）模型在大型状态/动作空间中进行规划，这是一种在多个抽象级别表示概率领域知识的新形式主义； （2）以AMDP的形式学习分层任务知识； (3) 通过映射到学习的层次结构来解释多个抽象级别的自然语言命令。该形式主义将在多个领域进行论证和验证，包括模拟“清理”玩具领域、具有挑战性和复杂性的视频游戏以及机器人操纵任务。

项目成果

Planning with Abstract Markov Decision Processes

使用抽象马尔可夫决策过程进行规划

• DOI: 10.1609/icaps.v27i1.13867
• 发表时间: 2017-06-05
• 作者: N. Gopalan;Marie desJardins;M. Littman;J. MacGlashan;S. Squire;Stefanie Tellex;J. Winder;Lawson L. S. Wong
• 通讯作者: Lawson L. S. Wong

Policy and Value Transfer in Lifelong Reinforcement Learning

终身强化学习中的政策和价值转移

• DOI: 10.1108/fs-04-2018-0034
• 发表时间: 2018-07-03
• 期刊: foresight
• 影响因子: 2
• 作者: David Abel;Yuu Jinnai;Yue (Sophie) Guo;G. Konidaris;M. Littman
• 通讯作者: M. Littman

State Abstractions for Lifelong Reinforcement Learning

终身强化学习的状态抽象

• 发表时间: 2018-07-03
• 作者: David Abel;Dilip Arumugam;Lucas Lehnert;M. Littman
• 通讯作者: M. Littman

The Expected-Length Model of Options

期权的预期长度模型

• 发表时间: 2019-01
• 期刊: IJCAI
• 作者: David Abel*; John Winder*
• 通讯作者: John Winder*

Near Optimal Behavior via Approximate State Abstraction

通过近似状态抽象实现近乎最优的行为

• 发表时间: 2016-06-19
• 作者: David Abel;D. E. Hershkowitz;M. Littman
• 通讯作者: M. Littman