CAREER: Foundations of Scalable and Resilient Distributed Real-Time Decision Making in Open Multi-Agent Systems

职业：开放多代理系统中可扩展和弹性分布式实时决策的基础

• 批准号: 2339509
• 负责人: Thinh Doan
• 金额: $ 54.99万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-03-01 至 2029-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2339509&HistoricalAwards=false
• 关键词: CAREER; Foundations; Scalable; Resilient; Distributed

Advances in artificial intelligence and machine learning provide the opportunity for using autonomous multi-agent systems to solve important social and economic problems, such as the application of multiple robots in wildfire monitoring, search-and-rescue, manufacturing, etc. In these systems, agents autonomously cooperate to make decisions in real-time to perform complex tasks. Reinforcement learning, a data-driven control method that enables agents to autonomously learn desired tasks by interacting directly with the environment, has emerged as one of the predominant frameworks for this kind of real-time decision making. While reinforcement learning provides a powerful and flexible framework, it suffers fundamental challenges in its scalability and resilience. Specifically, existing methods require a vast amount of data and computational power and can be unstable in the presence of various types of errors and adversaries. These challenges are the main barriers to the wide applicability of reinforcement learning for real-world problems. This CAREER project will develop new foundations of scalable and resilient distributed reinforcement learning for real-time autonomous cooperation in open multi-agent systems. The overarching goal is to design new learning and control methods that enable agents to interact effectively in open systems, adapt gracefully in time-varying environments, and be resilient to unexpected failures and adversaries. The project will also contribute to education and workforce development by integrating the research findings with rigorous educational and outreach activities, course development, student training, and public partnerships.The central idea of this project is to establish new fundamentals of two-time-scale stochastic approximation for non-monotone systems. The key approach is to leverage extrapolation techniques in optimization and singular perturbation theories in control to address the instability issues of stochastic approximation under non-monotone settings. New theoretical principles will be studied to characterize the finite-time complexity of the proposed methods. By leveraging these new results of two-time-scale stochastic approximation, this project will advance several foundational aspects of distributed learning and control in open multi-agent systems. The focus is to develop new scalable and resilient distributed multi-time-scale reinforcement learning methods that allow agents to cooperate efficiently in real-time under diverse practical considerations, including time-varying numbers of agents, unexpected failures, communication constraints, and adversaries. During the course of this project, the proposed research activities will be evaluated systematically through a series of simulations and field experiments of multi-robot navigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能和机器学习的进步为使用自主多智能体系统解决重要的社会和经济问题提供了机会，例如多个机器人在野火监测、搜索救援、制造等方面的应用。在这些系统中，智能体自主合作，实时做出决策，以执行复杂的任务。强化学习是一种数据驱动的控制方法，使代理能够通过直接与环境交互来自主学习所需的任务，已成为此类实时决策的主要框架之一。虽然强化学习提供了一个强大而灵活的框架，但它在可扩展性和弹性方面面临着根本性的挑战。具体来说，现有方法需要大量数据和计算能力，并且在存在各种类型的错误和对手的情况下可能不稳定。这些挑战是强化学习广泛应用于现实世界问题的主要障碍。该职业项目将为开放多智能体系统中的实时自主合作开发可扩展且有弹性的分布式强化学习的新基础。总体目标是设计新的学习和控制方法，使代理能够在开放系统中有效交互，在时变环境中优雅地适应，并对意外故障和对手具有弹性。该项目还将通过将研究结果与严格的教育和推广活动、课程开发、学生培训和公共合作伙伴关系相结合，为教育和劳动力发展做出贡献。该项目的中心思想是建立两倍尺度随机变量的新基础。非单调系统的近似。关键方法是利用优化中的外推技术和控制中的奇异摄动理论来解决非单调设置下随机逼近的不稳定性问题。将研究新的理论原理来表征所提出方法的有限时间复杂性。通过利用两倍尺度随机逼近的这些新结果，该项目将推进开放多智能体系统中分布式学习和控制的几个基础方面。重点是开发新的可扩展且有弹性的分布式多时间尺度强化学习方法，使智能体能够在不同的实际考虑因素下实时有效地合作，包括随时间变化的智能体数量、意外故障、通信限制和对手。在该项目过程中，将通过一系列多机器人导航的模拟和现场实验对拟议的研究活动进行系统评估。该奖项反映了 NSF 的法定使命，并通过利用基金会的智力优势和技术进行评估，认为值得支持。更广泛的影响审查标准。