Distributionally Robust Adaptive Control: Enabling Safe and Robust Reinforcement Learning

分布式鲁棒自适应控制：实现安全鲁棒的强化学习

基本信息

批准号：
2135925
负责人：
Naira Hovakimyan
金额：
$ 37.5万
依托单位：
University of Illinois at Urbana-Champaign
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-07-01 至 2025-06-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2135925&HistoricalAwards=false
关键词：
Distributionally Robust Adaptive Control Enabling

项目摘要

Data-driven algorithms can autonomously control complex systems like autonomous cars and drones. However, the use of such powerful algorithms remains relegated primarily to controlled laboratory environments. The main reason for the minimal adoption of data-driven methods for safety-critical systems is the difficulty one encounters when attempting to establish safety and predictability guarantees as one would do with well-established control theoretical methods. This award supports fundamental research to identify the best methodologies to consolidate data-driven and control-theoretic tools so that the overall methodology is safe, robust, and high-performing. The new approach lifts control tools to speak the same language as the data-driven methods. In doing so, the performance of the data-driven methods is not compromised, and yet, the safety guarantees of control-theoretic tools can be constructed. Safe and predictable autonomous operation of complex systems can bring immense socio-economic benefits through its application in medical robotics, autonomous logistics, transportation, and extra-terrestrial exploration, to name a few. This research involves multiple disciplines, including robotics, control theory, statistical learning, and mathematics. The cross-disciplinary nature will assist underrepresented groups' broader participation in STEM and impact engineering education. To adopt data-driven methods that rely on reinforcement learning (RL) algorithms in safety-critical systems, we need guarantees on safety and robustness. Robust and adaptive control methodologies developed for classical systems with parametric uncertainties cannot be used directly in conjunction with RL because the latter operates on data-driven models for which identifying parametric and deterministic uncertainties is difficult, if not impossible. This research will construct a new class of robust adaptive controllers that are robust to errors in the learned distributions, thus allowing RL algorithms to directly interact with these controllers without further restrictions. Due to robustness at the level of distributions, notions of risk-aware safety can be included in a straightforward manner. This research will first aim to construct controllers that track temporally evolving state distributions with uniform bounds. Then, the epistemic uncertainties will be introduced with a novel adaptive control scheme to quantifiably control the effect of the uncertainties in the space of distributions. The results produced through this effort will bring the two distinct worlds of data-driven control and classical control together at a natural intersection point where trajectories of distributions, not of sample paths, are considered.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据驱动的算法可以自主控制自动驾驶汽车和无人机等复杂系统。然而，如此强大的算法的使用仍然主要局限于受控的实验室环境。安全关键系统很少采用数据驱动方法的主要原因是，在尝试建立安全性和可预测性保证时遇到困难，就像使用完善的控制理论方法一样。该奖项支持基础研究，以确定整合数据驱动和控制理论工具的最佳方法，从而使整体方法安全、稳健且高性能。新方法将控制工具提升为与数据驱动方法相同的语言。这样做，数据驱动方法的性能不会受到影响，而且可以构建控制理论工具的安全保证。复杂系统的安全且可预测的自主运行可以通过其在医疗机器人、自主物流、运输和外星探索等领域的应用带来巨大的社会经济效益。这项研究涉及多个学科，包括机器人学、控制理论、统计学习和数学。跨学科性质将有助于代表性不足的群体更广泛地参与 STEM 并影响工程教育。为了在安全关键系统中采用依赖强化学习（RL）算法的数据驱动方法，我们需要保证安全性和鲁棒性。为具有参数不确定性的经典系统开发的鲁棒自适应控制方法不能直接与强化学习结合使用，因为后者在数据驱动模型上运行，而识别参数和确定性不确定性即使不是不可能，也是很困难的。这项研究将构建一类新的鲁棒自适应控制器，它们对学习分布中的错误具有鲁棒性，从而允许强化学习算法直接与这些控制器交互，而不受进一步的限制。由于分布级别的稳健性，可以以简单的方式包含风险意识安全的概念。这项研究首先旨在构建能够跟踪具有统一边界的时间演化状态分布的控制器。然后，将通过一种新颖的自适应控制方案引入认知不确定性，以量化控制分布空间中不确定性的影响。通过这项努力产生的结果将把数据驱动控制和经典控制这两个不同的世界结合在一个自然交叉点，其中考虑的是分布轨迹，而不是样本路径。该奖项反映了 NSF 的法定使命，并被认为是值得的通过使用基金会的智力优势和更广泛的影响审查标准进行评估来获得支持。