RI: Small: Model-Based Deep Reinforcement Learning for Domain Transfer

RI：小型：用于域迁移的基于模型的深度强化学习

• 批准号: 1614653
• 负责人: Sergey Levine
• 金额: $ 47.93万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2016-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1614653&HistoricalAwards=false
• 关键词: RI; Small; Model; Based; Deep

The goal of this project is to develop machine learning algorithms that can enable automated decision making and control in applications that require autonomous agents to interact with the real world. In particular, the project will examine two application areas: autonomous robots and educational agents that interact with human students to facilitate learning. The principal technical development investigated in this project will center around applications of deep neural networks (deep learning) to efficiently learn predictive models of the world, such as the physical environment of the robot or the behavior of a human student using an interactive educational agent. Deep learning has enabled impressive advances in passive perception domains such as computer vision and speech recognition, but typically requires very large amounts of data to succeed. This is often a major challenge in interactive settings, where a robot cannot interact with its environment for weeks or months just to learn a single behavior. To address this challenge, this project will investigate how predictive models can be transferred from prior tasks into a new task. The technologies developed as part of this project could enable substantially more sophisticated autonomous systems that can adapt quickly to new situations through transfer. Economic impact could include new consumer robotics products and improved education through intelligent automation.Reinforcement learning holds the promise of automating complex decision making and control in the presence of uncertainty. For a wide range of real-world problems, from robotic control and autonomous vehicles to interactive educational tools, this would provide dramatic improvements in capability and reduction in engineering cost. However, applying reinforcement learning to complex, unstructured environments and real-world problems with raw inputs, such as images and sounds, remains tremendously difficult. Deep learning has shown a great deal of promise for tackling complex learning problems, especially ones that require parsing high-dimensional, raw sensory signals, but the most successful applications of deep learning use very large amounts of labeled data. This is at odds with the demands of reinforcement learning, where the goal is typically to learn an effective policy using the minimal amount of interaction. This projects aims to address this challenge by developing algorithms for model-based deep reinforcement learning, where a generalizable model is learned from past experience on related but different tasks, and then transferred to a new task to learn it very quickly, directly using raw sensory inputs.

该项目的目的是开发机器学习算法，该算法可以在需要自主代理与现实世界互动的应用程序中实现自动决策和控制。特别是，该项目将检查两个应用领域：与人类学生互动以促进学习的自主机器人和教育代理商。该项目中调查的主要技术发展将围绕深度神经网络（深度学习）的应用，以有效地学习世界的预测模型，例如机器人的物理环境或使用互动教育代理人的人类学生的行为。深度学习使被动感知域（例如计算机视觉和语音识别）的令人印象深刻的进步，但通常需要大量数据才能成功。这通常是互动环境中的主要挑战，在互动环境中，机器人几周或几个月就无法与环境互动，只是为了学习一种行为。为了应对这一挑战，该项目将调查如何将预测模型从先前的任务传输到新任务。作为该项目的一部分开发的技术可以实现基本更复杂的自主系统，这些系统可以通过转移迅速适应新情况。经济影响可能包括新的消费者机器人产品和通过智能自动化改善教育。强化学习有望在不确定性的情况下自动化复杂的决策和控制。对于各种现实世界中的问题，从机器人控制和自动驾驶汽车到交互式教育工具，这将为工程成本的能力和降低提供巨大的改善。但是，将加固学习应用于复杂的，非结构化的环境和原始输入（例如图像和声音）的现实问题仍然非常困难。深度学习对解决复杂的学习问题有很大的希望，尤其是那些需要解析高维，原始感官信号的问题，但是深度学习的最成功应用程序使用了大量标记的数据。这与强化学习的需求不一致，在这种需求中，通常是使用最少的互动来学习有效政策的目标。该项目旨在通过为基于模型的深入增强学习算法开发算法来应对这一挑战，在这些算法中，从相关但不同的任务的过去经验中学到了可推广的模型，然后转移到新任务中，直接使用原始感官直接学习它输入。