RI: Medium: Learning Task-Specific Representations for Broadly Capable Reinforcement Learning Agents

RI：中：学习具有广泛能力的强化学习代理的特定任务表示

• 批准号: 1955361
• 负责人: George Konidaris
• 金额: $ 119.97万
• 依托单位: Brown University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955361&HistoricalAwards=false
• 关键词: RI; Medium; Learning; Task; Specific

While artificially intelligent agents have achieved expert-level performance on some specialized tasks, progress on designing agents that are broadly capable---able to reach adequate performance on a wide range of tasks---remains elusive. One major obstacle is that the sensors and actuators required by a general-purpose agent must be very complex, to support all the different tasks it may be required to solve. The resulting complexity makes decision-making much harder and drastically hinders the effectiveness of such agents. By contrast, agents that do only one thing can be given much simpler inputs and outputs that are carefully designed to be low-dimensional, highly informative, and task-relevant; such agents often demonstrate satisfactory performance. This project posits that a key requirement for generally intelligent agents is the ability to autonomously formulate such representations for themselves---as abstactions over their complex sensor and actuator spaces---and plans to design new algorithms to do so. AI systems with this ability could be re-tasked to solve many different problems without modification, rather than requiring substantial (and often prohibitive) engineering effort for each new application.This project aims to develop new algorithms that enable agents to learn compact, task-specific abstractions of new problems, by combining and extending techniques for discovering high-level actions, discovering perceptual abstractions that support planning with high-level actions, and formally characterizing the complexity and value loss of using those abstractions. The project will: 1) design new algorithms for reward-driven (and therefore task-specific) perceptual- and action-abstraction discovery; 2) enable inter-task abstraction transfer (which avoids having to re-learn abstractions from scratch each time) through new algorithms for learning generalized skills and constructing modular action-perception-abstraction packages, and new theory characterizing the value loss of using such generalized abstractions; and 3) create principled methods for incrementally constructing a library of modular action-perception abstractions and for adaptively recruiting existing action-state abstractions to solve new tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

项目成果

Robustly Learning Composable Options in Deep Reinforcement Learning

• DOI: 10.24963/ijcai.2021/298
• 发表时间: 2021-08
• 作者: Akhil Bagaria;J. Senthil;Matthew Slivinski;G. Konidaris

Model-based Lifelong Reinforcement Learning with Bayesian Exploration

• DOI: 10.48550/arxiv.2210.11579
• 发表时间: 2022-10
• 期刊: ArXiv
• 作者: Haotian Fu;Shangqun Yu;Michael S. Littman;G. Konidaris

Skill Discovery for Exploration and Planning using Deep Skill Graphs

使用深度技能图进行探索和规划的技能发现

• 发表时间: 2021
• 期刊: Proceedings of the Thirty-Eighth International Conference on Machine Learning
• 作者: Bagaria, A;Senthil, J.;Konidaris, G.D.
• 通讯作者: Konidaris, G.D.

Coarse-Grained Smoothness for Reinforcement Learning in Metric Spaces

度量空间中强化学习的粗粒度平滑度

• 发表时间: 2023
• 期刊: Proceedings of the 26th International Conference on Artificial Intelligence and Statistics
• 作者: Gottesman, O;Asadi, K;Allen, C;Lobel, S;Konidaris, GD;Littman, ML
• 通讯作者: Littman, ML

Autonomous Learning of Object-Centric Abstractions for High-Level Planning

用于高层规划的以对象为中心的抽象的自主学习

• 发表时间: 2022
• 期刊: Proceedings of the The Tenth International Conference on Learning Representations
• 作者: James, S;Rosman, B;Konidaris, G.D.
• 通讯作者: Konidaris, G.D.