CAREER: Distributionally Robust Learning, Control, and Benefits Analysis of Information Sharing for Connected and Autonomous Vehicles

职业：互联和自动驾驶车辆信息共享的分布式鲁棒学习、控制和效益分析

• 批准号: 2047354
• 负责人: Fei Miao
• 金额: $ 50.96万
• 依托单位: University of Connecticut
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-06-01 至 2026-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2047354&HistoricalAwards=false
• 关键词: CAREER; Distributionally; Robust; Learning; Control

The rapid evolution of ubiquitous sensing, communication, and computation technologies has contributed to the revolution of cyber-physical systems (CPS). Learning-based methodologies are integrated to the control of physical systems and demonstrating impressive performance in many CPS domains and connected and autonomous vehicles (CAVs) system is one such example with the development of vehicle-to-everything communication technologies. However, existing literature still lacks understanding of the tridirectional relationship among communication, learning, and control. The main challenges to be solved include (1) how to model dynamic system state and state uncertainties with shared information, (2) how to make robust learning and control decisions under model uncertainties, (3) how to integrate learning and control to guarantee the safety of networked CPS, and (4) how to quantify the benefits of communication.To address these challenges, this CAREER proposal aims to design integrated communication, learning, and control rules that are robust to hybrid system model uncertainties for safe operation and system efficiency of CAVs. The key intellectual merit is the design of integrated distributionally robust multi-agent reinforcement learning (DRMARL) and control framework with rigorous safety guarantees, considering hybrid system state uncertainties predicted with shared information, and the development of scientific foundation for analyzing and quantifying the benefits of communication. The fundamental theory and algorithm principles will be validated using simulators, small-scale testbeds, and full-scale CAVs field demonstrations, to form a new framework for future connectivity, learning, and control of CAVs and networked CPS. The technical contributions are as follows. (1). With shared information, we will design a cooperative prediction algorithm to improve hybrid system state and model uncertainty representations needed by learning and control. (2). Given enhanced prediction, we will design an integrated DRMARL and control framework with rigorous safety guarantee, and a computationally tractable algorithm to calculate the hybrid system decision-making policy. This integrates the strengths of both learning and control to improve system safety and efficiency. (3). We will define formally and quantify the value of communication given and propose a novel learn to communicate approach, to utilize learning and control to improve the communication actions. This project will also integrate an educational plan with the research goals by developing a learning platform of ``ssCAVs'' as an education tool and new interdisciplinary courses on “learning and control”, undertaking outreach to the general public and K-12 students and teachers, and directly involving high-school scholars, undergraduate and graduate students in research. This project is in response to the NSF CAREER 20-525 solicitation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

无处不在的敏感性，通信和计算技术的快速演变有助于网络物理系统（CPS）的革命。基于学习的方法集成到对物理系统的控制，并在许多CPS域中证明了令人印象深刻的性能，并且连接和自动驾驶汽车（CAVS）系统就是一个这样的例子，即开发了所有设施通信技术。但是，现有文献仍然缺乏对沟通，学习和控制之间的三个方向关系的了解。 The main challenges to be solved include (1) how to model dynamic system state and state uncertainties with shared information, (2) how to make robust learning and control decisions under model uncertainties, (3) how to integrate learning and control to guarantee the safety of networked CPS, and (4) how to quantify the benefits of communication.To address these challenges, this CAREER proposal aims to design integrated communication, learning, and control rules that are robust to hybrid system model骑士的安全操作和系统效率的不确定性。关键的智力优点是设计综合分布强大的多代理增强学习（DRMARL）和具有严格安全保证的控制框架的设计，考虑到混合系统状态状态状态不确定性与共享信息预测，以及开发科学基础，用于分析和量化交流的收益。基本理论和算法原理将使用模拟器，小规模的测试床和全尺寸的CAVS现场演示进行验证，以形成一个新的框架，以实现未来的连通性，学习和控制CAVS和网络CPS的控制。技术贡献如下。 （1）。借助共享的信息，我们将设计一种合作预测算法，以改善学习和控制所需的混合系统状态和模型不确定性表示。 （2）。给定增强的预测，我们将设计一个具有严格安全保证的集成的DRMARL和控制框架，以及一种可计算障碍的算法来计算混合系统决策政策。这整合了学习和控制的优势，以提高系统的安全性和效率。 （3）。我们将正式定义并量化给定的交流的价值，并提出一种新颖的学习方法来交流方法，以利用学习和控制来改善交流行动。该项目还将通过开发一个“ SSCAVS”的学习平台作为教育工具和有关“学习和控制”的新跨学科课程，向公众和K-12学生和老师进行宣传，并直接涉及高中学者，研究生不足的研究生，将教育计划与研究目标整合在一起。该项目是对NSF职业20-525征集的回应。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响评估标准来评估，以诚实的支持。

项目成果

Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios

• DOI: 10.1109/icra48891.2023.10161216
• 发表时间: 2022-10
• 期刊: 2023 IEEE International Conference on Robotics and Automation (ICRA)
• 作者: Zhili Zhang;Songyang Han;Jiangwei Wang;Fei Miao

Stable and Efficient Shapley Value-Based Reward Reallocation for Multi-Agent Reinforcement Learning of Autonomous Vehicles

• DOI: 10.48550/arxiv.2203.06333
• 发表时间: 2022-03
• 期刊: 2022 International Conference on Robotics and Automation (ICRA)
• 作者: Songyang Han;He Wang;Sanbao Su;Yuanyuan Shi;Fei Miao

A Multi-Agent Reinforcement Learning Approach for Safe and Efficient Behavior Planning of Connected Autonomous Vehicles

• DOI: 10.1109/tits.2023.3336670
• 发表时间: 2020-03
• 期刊: IEEE Transactions on Intelligent Transportation Systems
• 影响因子: 8.5
• 作者: Songyang Han;Shangli Zhou;Jiangwei Wang;Lynn Pepin;Caiwen Ding;Jie Fu;Fei Miao

Robust Multi-Agent Reinforcement Learning with Adversarial State Uncertainties

具有对抗性状态不确定性的鲁棒多智能体强化学习

• 发表时间: 2023
• 期刊: Transactions on Machine Learning Research
• 作者: He, Sihong;Han, Songyang;Su, Sanbao;Han, Shuo;Zou, Shaofeng;Miao, Fei.
• 通讯作者: Miao, Fei.

Data-Driven Distributionally Robust Electric Vehicle Balancing for Autonomous Mobility-on-Demand Systems Under Demand and Supply Uncertainties

• DOI: 10.1109/tits.2023.3237804
• 发表时间: 2022-11
• 期刊: IEEE Transactions on Intelligent Transportation Systems
• 影响因子: 8.5
• 作者: Sihong He;Zhili Zhang;Shuo Han;Lynn Pepin;Guang Wang;Desheng Zhang;J. Stankovic;Fei Miao