RI: Small: Model-Based Deep Reinforcement Learning for Domain Transfer

RI：小型：用于域迁移的基于模型的深度强化学习

• 批准号: 1700697
• 负责人: Sergey Levine
• 金额: $ 47.93万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1700697&HistoricalAwards=false
• 关键词: RI; Small; Model; Based; Deep

The goal of this project is to develop machine learning algorithms that can enable automated decision making and control in applications that require autonomous agents to interact with the real world. In particular, the project will examine two application areas: autonomous robots and educational agents that interact with human students to facilitate learning. The principal technical development investigated in this project will center around applications of deep neural networks (deep learning) to efficiently learn predictive models of the world, such as the physical environment of the robot or the behavior of a human student using an interactive educational agent. Deep learning has enabled impressive advances in passive perception domains such as computer vision and speech recognition, but typically requires very large amounts of data to succeed. This is often a major challenge in interactive settings, where a robot cannot interact with its environment for weeks or months just to learn a single behavior. To address this challenge, this project will investigate how predictive models can be transferred from prior tasks into a new task. The technologies developed as part of this project could enable substantially more sophisticated autonomous systems that can adapt quickly to new situations through transfer. Economic impact could include new consumer robotics products and improved education through intelligent automation.Reinforcement learning holds the promise of automating complex decision making and control in the presence of uncertainty. For a wide range of real-world problems, from robotic control and autonomous vehicles to interactive educational tools, this would provide dramatic improvements in capability and reduction in engineering cost. However, applying reinforcement learning to complex, unstructured environments and real-world problems with raw inputs, such as images and sounds, remains tremendously difficult. Deep learning has shown a great deal of promise for tackling complex learning problems, especially ones that require parsing high-dimensional, raw sensory signals, but the most successful applications of deep learning use very large amounts of labeled data. This is at odds with the demands of reinforcement learning, where the goal is typically to learn an effective policy using the minimal amount of interaction. This projects aims to address this challenge by developing algorithms for model-based deep reinforcement learning, where a generalizable model is learned from past experience on related but different tasks, and then transferred to a new task to learn it very quickly, directly using raw sensory inputs.

该项目的目标是开发机器学习算法，可以在需要自主代理与现实世界交互的应用程序中实现自动决策和控制。该项目将特别研究两个应用领域：自主机器人和与人类学生互动以促进学习的教育代理。该项目研究的主要技术开发将围绕深度神经网络（深度学习）的应用来有效地学习世界的预测模型，例如机器人的物理环境或使用交互式教育代理的人类学生的行为。深度学习在计算机视觉和语音识别等被动感知领域取得了令人瞩目的进步，但通常需要大量数据才能取得成功。这通常是交互环境中的一个重大挑战，因为机器人无法为了学习单一行为而与环境交互数周或数月。为了应对这一挑战，该项目将研究如何将预测模型从先前的任务转移到新的任务中。作为该项目一部分开发的技术可以实现更加复杂的自主系统，这些系统可以通过传输快速适应新情况。经济影响可能包括新的消费机器人产品和通过智能自动化改善教育。强化学习有望在存在不确定性的情况下实现复杂决策和控制的自动化。对于广泛的现实问题，从机器人控制和自动驾驶车辆到交互式教育工具，这将显着提高功能并降低工程成本。然而，将强化学习应用于复杂的、非结构化的环境以及具有原始输入（例如图像和声音）的现实问题仍然非常困难。深度学习在解决复杂的学习问题方面展现出了巨大的前景，特别是那些需要解析高维、原始感官信号的问题，但深度学习最成功的应用使用了大量的标记数据。这与强化学习的要求不一致，强化学习的目标通常是使用最少的交互来学习有效的策略。该项目旨在通过开发基于模型的深度强化学习算法来应对这一挑战，其中可概括的模型是从过去相关但不同任务的经验中学习的，然后转移到新任务以直接使用原始感官快速学习输入。