CAREER: Foundations of Scalable and Resilient Distributed Real-Time Decision Making in Open Multi-Agent Systems

职业：开放多代理系统中可扩展和弹性分布式实时决策的基础

• 批准号: 2339509
• 负责人: Thinh Doan
• 金额: $ 54.99万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-03-01 至 2029-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2339509&HistoricalAwards=false
• 关键词: CAREER; Foundations; Scalable; Resilient; Distributed

Advances in artificial intelligence and machine learning provide the opportunity for using autonomous multi-agent systems to solve important social and economic problems, such as the application of multiple robots in wildfire monitoring, search-and-rescue, manufacturing, etc. In these systems, agents autonomously cooperate to make decisions in real-time to perform complex tasks. Reinforcement learning, a data-driven control method that enables agents to autonomously learn desired tasks by interacting directly with the environment, has emerged as one of the predominant frameworks for this kind of real-time decision making. While reinforcement learning provides a powerful and flexible framework, it suffers fundamental challenges in its scalability and resilience. Specifically, existing methods require a vast amount of data and computational power and can be unstable in the presence of various types of errors and adversaries. These challenges are the main barriers to the wide applicability of reinforcement learning for real-world problems. This CAREER project will develop new foundations of scalable and resilient distributed reinforcement learning for real-time autonomous cooperation in open multi-agent systems. The overarching goal is to design new learning and control methods that enable agents to interact effectively in open systems, adapt gracefully in time-varying environments, and be resilient to unexpected failures and adversaries. The project will also contribute to education and workforce development by integrating the research findings with rigorous educational and outreach activities, course development, student training, and public partnerships.The central idea of this project is to establish new fundamentals of two-time-scale stochastic approximation for non-monotone systems. The key approach is to leverage extrapolation techniques in optimization and singular perturbation theories in control to address the instability issues of stochastic approximation under non-monotone settings. New theoretical principles will be studied to characterize the finite-time complexity of the proposed methods. By leveraging these new results of two-time-scale stochastic approximation, this project will advance several foundational aspects of distributed learning and control in open multi-agent systems. The focus is to develop new scalable and resilient distributed multi-time-scale reinforcement learning methods that allow agents to cooperate efficiently in real-time under diverse practical considerations, including time-varying numbers of agents, unexpected failures, communication constraints, and adversaries. During the course of this project, the proposed research activities will be evaluated systematically through a series of simulations and field experiments of multi-robot navigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能和机器学习的进步为使用自主多代理系统解决重要的社会和经济问题提供了机会，例如在野火监控，搜索，搜索，制造，制造等中应用多个机器人。在这些系统中，代理商自主合作以实时做出决策以执行复杂的任务。增强学习是一种数据驱动的控制方法，使代理可以通过直接与环境进行交互自主学习所需的任务，它已成为这种实时决策的主要框架之一。强化学习提供了一个强大而灵活的框架，但它在其可伸缩性和弹性方面遇到了基本挑战。具体而言，现有方法需要大量的数据和计算能力，并且在存在各种类型的错误和对手的情况下可能是不稳定的。这些挑战是强化学习在现实世界问题上的广泛适用性的主要障碍。这个职业项目将开发新的基础，以开放多代理系统的实时自主合作为基础，以扩展和弹性的分布式增强学习。总体目标是设计新的学习和控制方法，使代理能够在开放系统中有效互动，在时间变化的环境中优雅地适应，并具有弹性的意外故障和对手。该项目还将通过将研究结果与严格的教育和外展活动，课程发展，学生培训和公共合作伙伴关系整合在一起来为教育和劳动力发展做出贡献。该项目的核心思想是建立针对非单调系统的两次尺度随机近似的新基础。关键方法是利用控制中的优化和奇异扰动理论中的外推技术来解决非单调设置下随机近似的不稳定性问题。将研究新的理论原理，以表征所提出方法的有限时间复杂性。通过利用两次尺度随机近似的这些新结果，该项目将推动开放多代理系统中分布式学习和控制的几个基本方面。重点是开发新的可扩展和弹性的分布式多时间加强学习方法，以使代理在不同的实际考虑因素下实时有效合作，包括时间变化的代理，意外的失败，沟通约束和对手。在该项目的过程中，拟议的研究活动将通过一系列的模拟和多机车导航的现场实验来系统地评估。该奖项反映了NSF的法定任务，并被认为值得通过基金会的知识分子优点和更广泛的影响来通过评估来进行支持。