CAREER: Decision Procedures for High-Assurance, AI-Controlled, Cyber-Physical Systems

职业：高可信度、人工智能控制、网络物理系统的决策程序

• 批准号: 2002405
• 负责人: Yasser Shoukry
• 金额: $ 49.69万
• 依托单位: University of California-Irvine
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2002405&HistoricalAwards=false
• 关键词: CAREER; Decision; Procedures; Assurance; AI

This project explores new mathematical techniques that provide a scientific basis to understand the fundamental properties of Cyber-Physical Systems (CPS) controlled by Artificial Intelligence (AI) and guide their design. From simple logical constructs to complex deep neural network models, AI agents are increasingly controlling physical/mechanical systems. Self-driving cars, drones, and smart cities are just examples of AI-controlled CPS. However, regardless of the explosion in the use of AI within a multitude of CPS domains, the safety and reliability of these AI-controlled CPS is still an under-studied problem. This project includes activities integrated with education, so as to explore how learning through counterexamples works for AI, and to help with critical thinking skills for young students.This project investigates a new generation of formal method tools that are capable of simultaneously analyzing the cyber components (including AI-agents) and the physical components of CPS. This new generation of formal methods will be used to analyze the safety and reliability of AI-controlled CPS, characterize the environments for which the system is guaranteed to operate correctly and predict their failure at real-time. In addition, this project will address the problem of how to assign the blame of system failures in AI-controlled CPS. The proposed methods will be evaluated over two flagship testbeds (i) autonomous drones and (ii) self-driving cars. The project engages graduate, undergraduate, and high-school students, and reaches out to the scientific community by providing open source implementations of algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目探索新的数学技术，为理解人工智能 (AI) 控制的网络物理系统 (CPS) 的基本属性并指导其设计提供科学基础。从简单的逻辑结构到复杂的深度神经网络模型，人工智能代理越来越多地控制物理/机械系统。自动驾驶汽车、无人机和智能城市只是人工智能控制的 CPS 的例子。然而，尽管人工智能在众多 CPS 领域中的使用呈爆炸式增长，但这些人工智能控制的 CPS 的安全性和可靠性仍然是一个尚未得到充分研究的问题。该项目包括与教育相结合的活动，以探索通过反例学习如何为人工智能发挥作用，并帮助年轻学生培养批判性思维技能。该项目研究能够同时分析网络组件的新一代形式方法工具（包括人工智能代理）和 CPS 的物理组件。新一代形式化方法将用于分析人工智能控制的CPS的安全性和可靠性，表征保证系统正确运行的环境并实时预测其故障。此外，该项目还将解决人工智能控制的CPS中系统故障的责任如何分配的问题。所提出的方法将在两个旗舰测试平台（i）自主无人机和（ii）自动驾驶汽车上进行评估。该项目吸引了研究生、本科生和高中生，并通过提供算法的开源实现来接触科学界。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的评估进行评估，认为值得支持。影响审查标准。

项目成果

Safe-by-Repair: A Convex Optimization Approach for Repairing Unsafe Two-Level Lattice Neural Network Controllers

修复安全：修复不安全两级格子神经网络控制器的凸优化方法

• DOI: 10.1109/cdc51059.2022.9993239
• 发表时间: 2022-12
• 期刊: Safe-by-Repair: A Convex Optimization Approach for Repairing Unsafe Two-Level Lattice Neural Network Controllers
• 作者: Cruz, Ulices Santa;Ferlez, James;Shoukry, Yasser
• 通讯作者: Shoukry, Yasser

Provably Safe Model-Based Meta Reinforcement Learning: An Abstraction-Based Approach

可证明安全的基于模型的元强化学习：一种基于抽象的方法

• DOI: 10.1109/cdc45484.2021.9683009
• 发表时间: 2021-12
• 期刊: 2021 60th IEEE Conference on Decision and Control (CDC
• 作者: Sun, Xiaowu;Fatnassi, Wael;Cruz, Ulices Santa;Shoukry, Yasser
• 通讯作者: Shoukry, Yasser

Polynomial-Time Reachability for LTI Systems With Two-Level Lattice Neural Network Controllers

具有两级晶格神经网络控制器的 LTI 系统的多项式时间可达性

• DOI: 10.1109/lcsys.2022.3231556
• 发表时间: 2022-09-20
• 期刊: IEEE Control Systems Letters
• 影响因子: 3
• 作者: James Ferlez;Yasser Shoukry
• 通讯作者: Yasser Shoukry

PEREGRiNN: Penalized-Relaxation Greedy Neural Network Verifier

PEREGRiNN：惩罚松弛贪婪神经网络验证器

• DOI: 10.1007/978-3-030-81685-8_13
• 发表时间: 2020-06-18
• 期刊: 2021 60th IEEE Conference on Decision and Control (CDC)
• 作者: Haitham Khedr;James Ferlez;Yasser Shoukry
• 通讯作者: Yasser Shoukry

AReN: assured ReLU NN architecture for model predictive control of LTI systems

AReN：用于 LTI 系统模型预测控制的有保证的 ReLU NN 架构

• DOI: 10.1145/3365365.3382213
• 发表时间: 2019-11-05
• 期刊: Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control
• 作者: James Ferlez;Yasser Shoukry
• 通讯作者: Yasser Shoukry