NSF-AoF: RI: Small: Safe Reinforcement Learning in Non-Stationary Environments With Fast Adaptation and Disturbance Prediction

NSF-AoF：RI：小型：具有快速适应和干扰预测功能的非平稳环境中的安全强化学习

基本信息

批准号：
2133656
负责人：
Naira Hovakimyan
金额：
$ 50万
依托单位：
University of Illinois at Urbana-Champaign
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-09-01 至 2024-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2133656&HistoricalAwards=false
关键词：
NSF AoF RI Small Safe

项目摘要

Reinforcement learning (RL) has shown impressive performance in the control of complex robotic systems for various tasks such as locomotion, manipulation, and playing sports, e.g., table tennis. Reinforcement learning enables a robot to autonomously discover an optimal behavior through trial-and-error interactions with its environment. However, the environmental perturbations could easily cause a behavior policy trained in an old environment to fail in a perturbed environment. The failure is unacceptable for safety-critical robotic systems such as self-driving cars, drones, flying taxies and construction machines. Existing robust methods try to consider all scenarios during the training phase and seek a fixed policy, leading to conservative behaviors. Existing adaptive methods try to update their behavior policies in the perturbed environment, but will only do that after the robot has “felt a difference” through its interaction with the environment. In contrast, a human could leverage his/her perception for prediction in the new environment and adjust his/her behavior accordingly even before interacting with it. In light of these conditions, this project envisions a new framework for safe and efficient RL in the presence of environmental changes leveraging fast adaptation and perception-based prediction. The framework will enable robotic and autonomous systems robustly and safely operate, learn and adapt in the real world. This project relies on the following thrusts: i) hybrid RL for safe and efficient policy updates, ii) robust adaptive control with safety guarantees; iii) vision-based disturbance prediction. More specifically, the project will develop robust adaptive control algorithms that ensure that the executed trajectory of a robot remains safe in the presence of disturbances induced by environmental changes. It will spur hybrid model-free/model-based RL algorithms that are capable of efficiently and safely updating the behavior policies with the help of the control algorithms. The project will advance novel methodologies for predicting the key parameters of the disturbances (e.g., the weight of a package) directly from the image observations, leading to new scalable methods for efficiently learning the mathematical model of the disturbances with quantified error bounds. All the ingredients will be holistically integrated to build a framework to enable robots to safely, robustly, and efficiently operate and adapt in real-world environments. Aerial and ground vehicles will be used for experimental validation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

强化学习 (RL) 在控制复杂机器人系统的各种任务方面表现出了令人印象深刻的性能，例如运动、操纵和体育运动（例如乒乓球）。强化学习使机器人能够通过反复试验自主发现最佳行为。然而，环境扰动很容易导致在旧环境中训练的行为策略在扰动环境中失败，这种失败对于自动驾驶汽车、无人机、飞行等安全关键型机器人系统来说是不可接受的。现有的鲁棒方法试图在训练阶段考虑所有场景并寻求固定的策略，导致现有的自适应方法尝试在扰动环境中更新其行为策略，但只会在机器人之后这样做。相比之下，人类可以利用他/她的感知在新环境中进行预测，并根据这些条件相应地调整他/她的行为。项目设想了一个安全高效的新框架在环境变化的情况下，利用快速适应和基于感知的预测，该框架将使机器人和自主系统能够在现实世界中稳健、安全地运行、学习和适应。安全高效的策略更新，ii）具有安全保证的鲁棒自适应控制；iii）基于视觉的干扰预测更具体地说，该项目将开发鲁棒的自适应控制算法，以确保机器人的执行轨迹在存在干扰的情况下保持安全。它将刺激混合动力。无模型/基于模型的强化学习算法能够在控制算法的帮助下高效、安全地更新行为策略，该项目将推进预测干扰关键参数（例如包裹的重量）的新方法。）直接从图像观察中产生，从而产生有效的新的可扩展方法，用于学习具有量化误差范围的干扰的数学模型。所有成分将被整体集成以构建一个框架，使机器人能够安全、稳健、高效地操作和适应。在空中和地面车辆将用于实验验证。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（9）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Tube-Certified Trajectory Tracking for Nonlinear Systems With Robust Control Contraction Metrics

具有鲁棒控制收缩指标的非线性系统的管认证轨迹跟踪

DOI：
10.1109/lra.2022.3153712
发表时间：
2022-04
期刊：
IEEE Robotics and Automation Letters
影响因子：
5.2
作者：
Zhao, Pan;Lakshmanan, Arun;Ackerman, Kasey;Gahlawat, Aditya;Pavone, Marco;Hovakimyan, Naira
通讯作者：
Hovakimyan, Naira

Convex Synthesis of Control Barrier Functions Under Input Constraints

输入约束下控制屏障函数的凸综合

DOI：
10.1109/lcsys.2023.3293765
发表时间：
2024-09-13
期刊：
IEEE Control Systems Letters
影响因子：
3
作者：
Pan Zhao;R. Ghabcheloo;Yikun Cheng;Hossein Abdi;N. Hovakimyan
通讯作者：
N. Hovakimyan

Simultaneous Spatial and Temporal Assignment for Fast UAV Trajectory Optimization Using Bilevel Optimization

使用双层优化同时进行空间和时间分配以实现快速无人机轨迹优化

DOI：
10.1109/lra.2023.3273399
发表时间：
2022-11-29
期刊：
IEEE Robotics and Automation Letters
影响因子：
5.2
作者：
Qianzhong Chen;Sheng Cheng;N. Hovakimyan
通讯作者：
N. Hovakimyan

Safe and Efficient Reinforcement Learning using Disturbance-Observer-Based Control Barrier Functions

使用基于干扰观察器的控制屏障函数进行安全高效的强化学习

DOI：
10.1109/restcon60981.2024.10463578
发表时间：
2022-11-30
期刊：
2024 1st International Conference on Robotics, Engineering, Science, and Technology (RESTCON)
影响因子：
0
作者：
Yikun Cheng;Pan Zhao;N. Hovakimyan
通讯作者：
N. Hovakimyan

Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functions

使用基于干扰观察器的控制屏障函数进行安全高效的强化学习