Career: IIS: RI: Improving Multi-Agent Reinforcement Learning for Cooperative, Partially Observable Settings

职业：IIS：RI：改进合作、部分可观察设置的多智能体强化学习

• 批准号: 2044993
• 负责人: Christopher Amato
• 金额: $ 55万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-03-01 至 2026-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2044993&HistoricalAwards=false
• 关键词: Career; IIS; RI; Improving; Multi

As intelligent systems become more prevalent, these systems will need to coordinate with each other (e.g., apps, robots, autonomous cars), resulting in multi-agent systems. Allowing multi-agent systems to learn will let them operate in more complex and realistic scenarios by adapting their behavior to fit specific needs. Reinforcement learning is a promising form of trial-and-error learning that has the potential to drastically improve outcomes in many multi-agent domains (e.g., warehouses, delivery), but new methods are required for coordinating the agents in realistic domains with noisy and limited communication and sensing (i.e., partial observability). This project will develop these new reinforcement learning methods for coordinating teams of agents in various partially observable settings. The results will impact the development of future artificial intelligence (AI) and robotic systems and will be conveyed through outreach and educational activities.This project will develop a number of novel methods for cooperative multi-agent reinforcement learning (MARL) under partial observability. MARL, the extension of reinforcement learning methods for multi-agent domains, has gained popularity for generating high-quality solutions in some domains, but more work is needed to make the methods more scalable and widely applicable. Therefore, this project will first provide a better theoretical understanding of centralized training for decentralized execution methods. Centralized training for decentralized execution is the dominant paradigm in MARL where agents are trained offline and only executed online. The project will then develop new centralized training methods that are unbiased, scalable and perform well in a wide range of domains. Second, the project will develop online decentralized learning methods that allow agents to learn online even in noisy multi-agent settings. Lastly, to allow agents to learn and execute in an asynchronous manner, the project will develop methods for asynchronous MARL as well as asynchronous hierarchical learning with learning over multiple layers of a hierarchy. The resulting methods will significantly improve performance, stability and scalability of MARL methods and make them more generally applicable to large realistic domains.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着智能系统变得越来越普遍，这些系统将需要相互协调（例如应用程序、机器人、自动驾驶汽车），从而产生多智能体系统。允许多智能体系统学习将使它们通过调整其行为以满足特定需求来在更复杂和现实的场景中运行。强化学习是一种有前途的试错学习形式，有可能极大地改善许多多智能体领域（例如仓库、交付）的结果，但需要新的方法来协调具有噪声和干扰的现实领域中的智能体。有限的通信和感知（即部分可观察性）。该项目将开发这些新的强化学习方法，用于在各种部分可观察的环境中协调代理团队。 研究结果将影响未来人工智能（AI）和机器人系统的发展，并将通过外展和教育活动进行传达。该项目将开发一些在部分可观测性下进行协作多智能体强化学习（MARL）的新颖方法。 MARL 是多智能体领域强化学习方法的扩展，由于在某些领域生成高质量的解决方案而受到欢迎，但需要做更多的工作来使该方法更具可扩展性和更广泛的适用性。因此，该项目将首先为去中心化执行方法提供对中心化训练更好的理论理解。分散执行的集中训练是 MARL 中的主导范例，其中代理离线训练，仅在线执行。然后，该项目将开发新的集中培训方法，这些方法是公正的、可扩展的并且在广泛的领域中表现良好。 其次，该项目将开发在线分散学习方法，使智能体即使在嘈杂的多智能体环境中也能在线学习。最后，为了允许代理以异步方式学习和执行，该项目将开发异步 MARL 以及异步分层学习的方法，并在层次结构的多个层上进行学习。由此产生的方法将显着提高 MARL 方法的性能、稳定性和可扩展性，并使它们更普遍地适用于大型现实领域。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learning

对比多智能体强化学习中的集中式和分散式批评

• DOI: 10.5555/3463952.3464053
• 发表时间: 2021-02-08
• 期刊: ArXiv
• 作者: Xueguang Lyu;Yuchen Xiao;Brett Daley;Chris Amato
• 通讯作者: Chris Amato

Shield Decentralization for Safe Multi-Agent Reinforcement Learning

用于安全多智能体强化学习的屏蔽去中心化

• 发表时间: 2022-01
• 期刊: Advances in neural information processing systems
• 作者: Melcer, Daniel;Amato, Christopher;Tripakis, Stavros
• 通讯作者: Tripakis, Stavros

Local Advantage Actor-Critic for Robust Multi-Agent Deep Reinforcement Learning.

用于鲁棒多智能体深度强化学习的局部优势演员-评论家。

• 发表时间: 2021-01
• 期刊: the International Symposium on Multi-Robot and Multi-Agent Systems
• 作者: Yuchen Xiao; Xueguang Lyu
• 通讯作者: Xueguang Lyu

Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning

用于多智能体强化学习的异步 Actor-Critic

• DOI: 10.48550/arxiv.2209.10113
• 发表时间: 2022-09-20
• 期刊: ArXiv
• 作者: Yuchen Xiao;Weihao Tan;Chris Amato
• 通讯作者: Chris Amato

On Centralized Critics in Multi-Agent Reinforcement Learning

多智能体强化学习中的集中批评

• DOI: 10.1613/jair.1.14386
• 发表时间: 2023-05
• 期刊: Journal of Artificial Intelligence Research
• 影响因子: 5
• 作者: Lyu, Xueguang;Baisero, Andrea;Xiao, Yuchen;Daley, Brett;Amato, Christopher
• 通讯作者: Amato, Christopher