CAREER: Temporal Causal Reinforcement Learning and Control for Autonomous and Swarm Cyber-Physical Systems

职业：自治和群体网络物理系统的时间因果强化学习和控制

• 批准号: 2339774
• 负责人: Zhe Xu
• 金额: $ 54.98万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-03-01 至 2029-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2339774&HistoricalAwards=false
• 关键词: CAREER; Temporal; Causal; Reinforcement; Learning

Understanding the root cause of behavior is imperative for informed decision-making and preventing ineffective or biased policies. Currently, most AI-based learning and control modules embedded in cyber-physical systems (CPS) rely on statistical correlation rather than causality for decision-making. This not only results in incorrect decisions but also hinders the interpretability of learning, limiting transferability and scalability. This CAREER proposal aims to bridge the gap between causal inference and the growing capabilities of reinforcement learning (RL) in CPS. The proposed methods are transformative to a wide range of CPS applications, enabling more efficient and effective decision-making processes in autonomous and swarm CPS such as self-driving cars, drones, industrial robots, and swarm robots.This NSF CAREER proposal proposes a set of temporal causal RL and control approaches for CPS by leveraging the reasoning capabilities of temporal logics and causal diagrams in single-agent, multi-agent, and swarm system settings. The tools we develop will be implemented on multiple CPS testbeds and integrated with the proposed education plan. The proposed algorithms have the following unique and innovative features. Firstly, we will develop computationally efficient tools that can discover temporal causal knowledge from both observational and interventional data of a CPS in performing RL to improve the sampling efficiency and transferability. Secondly, we will develop multi-agent RL approaches for CPS in cooperative, non-cooperative, and incomplete information stochastic game environments where temporal causal knowledge is discovered in a distributed way for expediting RL. Lastly, we will develop scalable RL-based control methods for swarm systems utilizing temporal causal reasoning over agent-level features and swarm-level features such as densities and generalized moments. The education plan will impact the next generation of CPS and AI engineers and researchers through AI-assisted adaptive and interactive teaching, temporal-logic-based educational games, online interactive educational website design for temporal causal RL, and workshops and webinars with industrial partners.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

了解行为的根本原因对于明智的决策和预防无效或有偏见的政策至关重要。当前，大多数基于AI的学习和控制模块嵌入在网络物理系统中（CPS），依赖于统计相关性而不是因果关系来制定决策。这不仅会导致决策不正确，而且还阻碍了学习的解释性，限制可转移性和可扩展性。这项职业建议旨在弥合因果推断与CPS中增强学习能力（RL）增长能力之间的差距。 The proposed methods are transformative to a wide range of CPS applications, enabling more efficient and effective decision-making processes in autonomous and swarm CPS such as self-driving cars, drones, industrial robots, and swarm robots.This NSF CAREER proposal proposes a set of temporal causal RL and control approaches for CPS by leveraging the reasoning capabilities of temporal logics and causal diagrams in single-agent,多机构和群体系统设置。我们开发的工具将在多个CPS测试床上实施，并与拟议的教育计划集成。所提出的算法具有以下独特和创新的特征。首先，我们将开发计算高效的工具，这些工具可以从CPS的观察数据和介入数据中发现暂时性因果知识，以提高采样效率和可传递性。其次，我们将开发用于合作，非合作性和不完整信息随机游戏环境中CP的多代理RL方法，其中以分布式方式发现了为加快RL的分布式因果知识。最后，我们将利用代理级特征和群体级特征（例如密度和广义瞬间）开发基于RL RL的控制方法，用于群体系统。教育计划将通过AI辅助适应性和互动教学，基于时间逻辑的教育游戏，在线互动式教育网站设计，以及与工业合作伙伴的研讨会和网络研讨会来影响下一代CPS和AI工程师和研究人员，并影响了NSF的法定任务和审查企业的范围。