CPS: Medium: Collaborative Research: Developing Data-driven Robustness and Safety from Single Agent Settings to Stochastic Dynamic Teams: Theory and Applications

CPS：中：协作研究：从单代理设置到随机动态团队开发数据驱动的鲁棒性和安全性：理论与应用

• 批准号: 2240981
• 负责人: Vijay Subramanian
• 金额: $ 48万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-06-01 至 2026-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2240981&HistoricalAwards=false
• 关键词: CPS; Medium; Collaborative; Research; Developing

This Cyber-Physical Systems (CPS) project will make foundational methodological advances that enable safe and robust reinforcement learning (RL)-based control algorithmic solutions that are driven by problems in smart traffic signal control systems. Recent advances in computation, communication, storage, and sensing have led to a demand for data-driven learning-based decision-making and control in modern cyber-physical systems (CPSs), such as smart transportation systems. In such systems, decision-making agents need to operate safely and in a robust manner while working in complex environments with constraints that need to be respected. This project will develop foundational advances in robust RL solutions, and safe and constrained RL with provable guarantees by taking traffic signal control systems within smart transportation systems as our motivating CPS application and evaluation platform. This work will additionally focus on advancing curriculum development, recruitment of students from under-represented groups, involvement of undergraduate students in research, K-12 outreach, and also research community outreach via workshops, conference sessions, and seminars. The researchers will interface with companies and other stakeholders to communicate the results of the research as well as provide them with educational material on methodology. The technical approaches include: 1. Robust RL solutions incorporating model class knowledge, use of future predictions and robustness characterizations, and off-policy methods to address distributional shifts and data paucity arising from the use of a simulator/emulator or offline data; and 2. Efficient, safe, and constrained RL algorithms using model-free approaches and function-approximated methods, and also methods for partially-observed systems. To close the loop with the motivating CPS application, the RL algorithms will be evaluated in the context of traffic signal control via a comprehensive simulation-based evaluation using models of two instrumented sites.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该网络物理系统 (CPS) 项目将取得基础方法上的进步，从而实现由智能交通信号控制系统中的问题驱动的安全且稳健的基于强化学习 (RL) 的控制算法解决方案。计算、通信、存储和传感领域的最新进展导致了现代网络物理系统（CPS）（例如智能交通系统）中对数据驱动的基于学习的决策和控制的需求。在此类系统中，决策代理需要以安全、稳健的方式运行，同时在需要遵守约束的复杂环境中工作。该项目将通过将智能交通系统中的交通信号控制系统作为我们的激励性 CPS 应用和评估平台，在稳健的 RL 解决方案以及具有可证明保证的安全且受限的 RL 方面取得基础性进展。 这项工作还将侧重于推进课程开发、从代表性不足的群体中招收学生、本科生参与研究、K-12 外展以及通过讲习班、会议和研讨会进行研究社区外展。研究人员将与公司和其他利益相关者互动，传达研究结果，并为他们提供有关方法论的教育材料。技术方法包括： 1. 鲁棒强化学习解决方案，结合模型类知识、使用未来预测和鲁棒性特征，以及离策略方法来解决由于使用模拟器/仿真器或离线数据而引起的分布变化和数据匮乏问题； 2. 使用无模型方法和函数近似方法以及部分观测系统的方法的高效、安全和受限的强化学习算法。为了与激励 CPS 应用形成闭环，将通过使用两个仪表站点的模型进行基于模拟的综合评估，在交通信号控制的背景下对 RL 算法进行评估。该奖项反映了 NSF 的法定使命，并被认为值得支持通过使用基金会的智力优点和更广泛的影响审查标准进行评估。

项目成果

Learning-Based Optimal Admission Control in a Single-Server Queuing System

单服务器排队系统中基于学习的最优准入控制

• 发表时间: 2023-12
• 期刊: Stochastic systems
• 作者: Asaf Cohen;Vijay Subramanian;Yili Zhang
• 通讯作者: Yili Zhang

A Strong Duality Result for Cooperative Decentralized Constrained POMDPs

合作分散约束 POMDP 的强对偶结果

• 发表时间: 2023-12
• 期刊: Proceedings of the IEEE Conference on Decision Control
• 作者: Nouman Khan and Vijay Subramanian

Localization and Approximations for Distributed Non-convex Optimization

分布式非凸优化的本地化和近似

• DOI: 10.1007/s10957-023-02328-8
• 发表时间: 2023-11
• 期刊: Journal of Optimization Theory and Applications
• 影响因子: 1.9
• 作者: Kao, Hsu;Subramanian, Vijay
• 通讯作者: Subramanian, Vijay

Bayesian Learning of Optimal Policies in Markov Decision Processes with Countably Infinite State-Space

可数无限状态空间马尔可夫决策过程中最优策略的贝叶斯学习

• 发表时间: 2023-12
• 期刊: Advances in Neural Information Processing Systems 36 (NeurIPS 2023
• 作者: Saghar Adler;Vijay Subramanian
• 通讯作者: Vijay Subramanian

A Multi-Agent View of Wireless Video Streaming with Delayed Client-Feedback

具有延迟客户端反馈的无线视频流的多代理视图

• 发表时间: 2024-05
• 期刊: IEEE International Conference on Computer Communication and the Internet
• 作者: Nouman Khan;Ujwal Dinesha;Subrahmanyam Arunachalam;Dheeraj Narasimha;Vijay Subramanian;and Srinivas Shakkottai
• 通讯作者: and Srinivas Shakkottai