CRII: RI: Secure Multi-Agent Reinforcement Learning Algorithms

CRII：RI：安全多代理强化学习算法

• 批准号: 2105007
• 负责人: Sarra Alqahtani
• 金额: $ 17.49万
• 依托单位: Wake Forest University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-05-15 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2105007&HistoricalAwards=false
• 关键词: CRII; RI; Secure; Multi; Agent

Recent years have witnessed significant advances in reinforcement learning (RL), an area of machine learning that achieved great success in solving various sequential decision-making problems. Advances in single‐agent RL algorithms sparked new interest in multi-agent RL (MARL). The goal of this project is to build robust, secure algorithms for autonomous systems that are built using MARL. The project team will investigate a novel threat that can be exploited simply by designing an adversarial plan for an agent acting in a cooperative multi‐agent environment so as to create natural observations that are adversarial to one or more of its allies. For example, in connected autonomous vehicles, one compromised vehicle candrastically disrupt security, causing confusion and mistakes that result in poor performance and even harm to humans who rely on these systems. The project team will build a robust MARL algorithm to such adversarial manipulations. The educational plan for this project includes developing a suit of tutorials on analyzing the security and robustness of MARL algorithms, designed for use in a graduate course or as a tool for MARL researchers. The project team will also contribute to educational outreach by involving graduate and undergraduate students from underrepresented groups.The project is built upon three overarching objectives (1) study how attackers can exploit MARL vulnerabilities,(2) develop a more robust MARL algorithm by training each agent using the counterfactual reasoning about other agents’ behaviors, and (3) create a novel online formal verification method to satisfy the security and safety requirements during the execution of our proposed MARL algorithm. More specifically, for the first objective, the project team will prove the feasibility of using a compromised agent to attack its allies in MARL systems through its actions. The second objective will reverse‐engineer the attack strategies to develop a robust MARL algorithm that models the agents’ behaviors during training and correlates their actions using counterfactual reasoning. In the third objective, an online formal verification model will be developed to detect any deviations in agents’ behaviors using a predefined set of security and safety specifications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，强化学习 (RL) 取得了重大进展，这是机器学习的一个领域，在解决各种顺序决策问题方面取得了巨大成功，单智能体 RL 算法的进步引发了人们对多智能体 RL (MARL) 的新兴趣。该项目的目标是为使用 MARL 构建的自治系统构建强大、安全的算法，该项目团队将研究一种新的威胁，只需为在协作多方中运行的代理设计一个对抗计划即可利用该威胁。代理环境，以创建自然观察结果例如，在联网的自动驾驶汽车中，一辆受损的汽车可能会严重破坏安全性，导致混乱和错误，从而导致性能不佳，甚至对项目团队将构建的人类造成伤害。该项目的教育计划包括开发一套分析 MARL 算法的安全性和鲁棒性的教程，旨在用于研究生课程或作为 MARL 研究人员的工具。也做出贡献通过让来自代表性不足群体的研究生和本科生参与教育推广。该项目建立在三个总体目标之上：(1) 研究攻击者如何利用 MARL 漏洞，(2) 通过使用关于其他代理的反事实推理来训练每个代理，开发更强大的 MARL 算法(3) 创建一种新颖的在线形式验证方法，以满足我们提出的 MARL 算法执行过程中的安全性和安全性要求。更具体地说，对于第一个目标，项目团队将证明使用妥协的可行性。代理来攻击它第二个目标将对攻击策略进行逆向工程，以开发强大的 MARL 算法，该算法在训练期间对代理的行为进行建模，并使用反事实推理将其行为关联起来。将开发使用一组预定义的安全和安全规范来检测代理行为中的任何偏差。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。