CRII: RI: Secure Multi-Agent Reinforcement Learning Algorithms

CRII：RI：安全多代理强化学习算法

• 批准号: 2105007
• 负责人: Sarra Alqahtani
• 金额: $ 17.49万
• 依托单位: Wake Forest University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-05-15 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2105007&HistoricalAwards=false
• 关键词: CRII; RI; Secure; Multi; Agent

Recent years have witnessed significant advances in reinforcement learning (RL), an area of machine learning that achieved great success in solving various sequential decision-making problems. Advances in single‐agent RL algorithms sparked new interest in multi-agent RL (MARL). The goal of this project is to build robust, secure algorithms for autonomous systems that are built using MARL. The project team will investigate a novel threat that can be exploited simply by designing an adversarial plan for an agent acting in a cooperative multi‐agent environment so as to create natural observations that are adversarial to one or more of its allies. For example, in connected autonomous vehicles, one compromised vehicle candrastically disrupt security, causing confusion and mistakes that result in poor performance and even harm to humans who rely on these systems. The project team will build a robust MARL algorithm to such adversarial manipulations. The educational plan for this project includes developing a suit of tutorials on analyzing the security and robustness of MARL algorithms, designed for use in a graduate course or as a tool for MARL researchers. The project team will also contribute to educational outreach by involving graduate and undergraduate students from underrepresented groups.The project is built upon three overarching objectives (1) study how attackers can exploit MARL vulnerabilities,(2) develop a more robust MARL algorithm by training each agent using the counterfactual reasoning about other agents’ behaviors, and (3) create a novel online formal verification method to satisfy the security and safety requirements during the execution of our proposed MARL algorithm. More specifically, for the first objective, the project team will prove the feasibility of using a compromised agent to attack its allies in MARL systems through its actions. The second objective will reverse‐engineer the attack strategies to develop a robust MARL algorithm that models the agents’ behaviors during training and correlates their actions using counterfactual reasoning. In the third objective, an online formal verification model will be developed to detect any deviations in agents’ behaviors using a predefined set of security and safety specifications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，在加强学习（RL）方面取得了重大进展，这是机器学习领域，在解决各种顺序决策问题方面取得了巨大成功。单药RL算法的进步引发了对多代理RL（MARL）的新兴趣。该项目的目的是为使用MARL构建的自主系统构建坚固，安全的算法。项目团队将调查一种新的威胁，可以通过为在合作多代理环境中作用的代理商设计对抗性计划来探索，以创建对一个或多个盟友的对抗性的自然观察。例如，在互联的自动驾驶汽车中，一辆损害车辆candrastrastrastrastrastrastrastrastrast造成了困惑和错误，导致性能不佳，甚至对依靠这些系统的人类造成伤害。项目团队将对这种对抗性操纵建立强大的MARL算法。该项目的教育计划包括开发一套有关分析MARL算法的安全性和鲁棒性的教程，该算法的安全性和稳健性是为了在研究生课程中或作为MARL研究人员的工具。 The project team will also contribute to educational outreach by involving graduate and undergraduate students from underrepresented groups.The project is built upon three overarching objectives (1) study how attackers can exploit MARL vulnerabilities,(2) develop a more robust MARL algorithm by training each agent using the counterfactual reasoning about other agents’ behaviors, and (3) create a novel online formal verification method to satisfy the security and safety requirements during我们提出的MARL算法的执行。更具体地说，对于第一个目标而言，项目团队将证明使用受损的代理商通过其行动在MARL系统中攻击其盟友的可行性。第二个目标将逆转工程的攻击策略，以开发出强大的MARL算法，该算法在训练过程中对代理商的行为进行建模，并使用反事实推理将其行为相关联。在第三个目标中，将开发出在线正式验证模型，以使用预定义的安全性和安全性规范来检测代理行为中的任何出发。该奖项反映了NSF的法定任务，并认为值得通过基金会的知识分子优点和更广泛的影响标准通过评估来进行评估。