Realtime Search for Learning Autonomous Agents

实时搜索学习自主代理

基本信息

批准号：
06452402
负责人：
ISHIDA Toru
金额：
$ 4.86万
依托单位：
KYOTO UNIVERSITY
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (B)
财政年份：
1994
资助国家：
日本
起止时间：
1994 至 1996
项目状态：
已结题

来源：
https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-06452402/
关键词：
Realtime search learning agent problem solving 強化学習 TD(λ)Q-learning

项目摘要

Existing search algorithms can be divided into two classes : offline search such as A^<**>, and realtime search such as Real-Time-A^<**> (RTA^<**>) and Learning Real-Time-A^<**> (LRTA^<**>). Offline search completely examines every possible path to the goal state before executing that path, while realtime search makes each decision in a constant time, and commits its decision to the physical world. The problem solver eventually reaches the goal by repeating the cycle of planning and execution. Realtime search cannot guarantee to find an optimal solution, but can interleave planning and execution.This research focuses on extending realtime search algorithms for autonomous agents and for a multiagent world. Though realtime search provides an attractive framework for resource-bounded problem solving, the behavior of the problem solver is not rational enough for autonomous agents : the problem solver tends to perform superfluous actions before attaining the goal ; and the problem solver cannot utilize and improve previous experiments. Other problems are that though the algorithms interleave planning and execution, they cannot be directly applied to a multiagent world ; the problem solver cannot adapt to the dynamically changing goals ; and the problem solver cannot cooperatively solve problems with other problem solvers. We developed a series of new algorithms including Weighted Realtime Search, Realtime Search with Upper Bounds, Moving Target Search and Realtime Bidirectional Search to overcome the above problems.

现有的搜索算法可以分为两个类：离线搜索，例如a^<**>，以及实时搜索，例如实时a^<**>（rta^<**>），并学习实时 - A^<**>（lrta^<**>）。离线搜索在执行该路径之前彻底检查了通往目标状态的每一个可能的路径，而实时搜索则在恒定时间内使每个决策都做出了决策，并将其决定对物理世界提出。问题解决者最终通过重复计划和执行的周期来实现目标。实时搜索不能保证找到一个最佳解决方案，而可以交织计划和执行。这项研究重点是扩展实时搜索算法的自主代理和多种世界。尽管实时搜索为解决资源结合的问题解决提供了一个有吸引力的框架，但问题解决者的行为对于自主代理来说还不够理性：问题求解器倾向于在达到目标之前执行多余的动作；问题解决者无法利用并改善以前的实验。其他问题是，尽管算法交织计划和执行，但它们不能直接应用于多种世界。问题解决者无法适应动态变化的目标；问题解决者无法合作解决其他问题解决者的问题。我们开发了一系列新的算法，包括加权实时搜索，具有上限的实时搜索，移动目标搜索和实时双向搜索以克服上述问题。