CAREER: Structure Exploiting Multi-Agent Reinforcement Learning for Large Scale Networked Systems: Locality and Beyond

职业：为大规模网络系统利用多智能体强化学习的结构：局部性及其他

• 批准号: 2339112
• 负责人: Guannan Qu
• 金额: $ 50万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-01-01 至 2028-12-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2339112&HistoricalAwards=false
• 关键词: CAREER; Structure; Exploiting; Multi; Agent

This project aims to develop a suite of Multi-Agent Reinforcement Learning (MARL) algorithms for the control of networked systems, which are ubiquitous and play an indispensable role in advancing our modern society. Examples cut across a broad spectrum, including power/energy grid, transportation systems, networked robots, etc. The control and operation of such systems have long been a tremendous challenge, due to the complex interdependence between different components in the system, the increasing number of connected and interacting agents, and the increasing environmental uncertainties. The project will bring transformative change to how networked systems are controlled and operated, achieved by using MARL to design novel policies with order-of-magnitude improvement in efficiency and reliability under complex and uncertain environments. The intellectual merit focuses on a unique structure-exploiting approach for MARL. Most of the existing MARL approaches take a black-box view of the underlying system without utilizing the underlying structure and are known to have scalability and stability/safety issues. In contrast, real-world systems have rich structural properties that are readily available to exploit, e.g. the connectivity topology in the power grid. In light of the above structural properties, the project exploits the underlying structure to design scalable, stable, and safe MARL for large-scale networked systems. The proposed framework is not only theoretically sound but also widely applicable. To show the wide applicability of the approach, this project will demonstrate it on power systems and load balancing in multiserver systems. The broader impacts introduce a unique structure-exploiting perspective, that is machine learning should be integrated with the structural property of specific engineering applications. The project includes plans to broaden the impact of this perspective. Beyond the research itself, the project includes a synergistic education, diversity, and a broadening participation plan: (a) development of a new course at the intersection of machine learning and engineering networked systems; (b) K12 curriculum development and outreach activities through the CMU Gelfand Outreach program; and (c) undergraduates research through the SURF program.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在开发一套用于控制网络系统的多智能体强化学习（MARL）算法，这些算法无处不在，在推动现代社会发展中发挥着不可或缺的作用。例子涉及广泛，包括电力/能源网、交通系统、网络机器人等。由于系统中不同组件之间复杂的相互依赖关系，此类系统的控制和操作长期以来一直是一个巨大的挑战。相互关联和相互作用的主体的数量，以及日益增加的环境不确定性。 该项目将为网络系统的控制和操作方式带来革命性的变化，通过使用 MARL 设计新颖的策略来实现，在复杂和不确定的环境下，效率和可靠性将得到数量级的提高。 智力优势集中在 MARL 独特的结构开发方法上。大多数现有的 MARL 方法都采用底层系统的黑盒视图，而不利用底层结构，并且已知存在可扩展性和稳定性/安全性问题。相比之下，现实世界的系统具有丰富的结构属性，很容易被利用，例如电网中的连接拓扑。鉴于上述结构特性，该项目利用底层结构为大规模网络系统设计可扩展、稳定、安全的MARL。所提出的框架不仅理论上合理，而且具有广泛的适用性。为了展示该方法的广泛适用性，该项目将在多服务器系统中的电源系统和负载平衡上进行演示。 更广泛的影响引入了独特的结构利用视角，即机器学习应该与特定工程应用的结构特性相结合。该项目包括扩大这一观点影响的计划。除了研究本身之外，该项目还包括协同教育、多样性和扩大参与计划：(a) 开发机器学习和工程网络系统交叉点的新课程； (b) 通过卡内基梅隆大学格尔凡德外展计划开展 K12 课程开发和外展活动； (c) 本科生通过 SURF 计划进行研究。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。