CRII: CPS: Towards a Model-Based Reinforcement Learning Approach for Safe Operation of Distributed Energy Systems

CRII：CPS：面向分布式能源系统安全运行的基于模型的强化学习方法

基本信息

批准号：
1850206
负责人：
Dileep Kalathil
金额：
$ 17.2万
依托单位：
Texas A&M Engineering Experiment Station
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-05-01 至 2022-04-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1850206&HistoricalAwards=false
关键词：
CRII CPS Towards Model Based

项目摘要

With the increasing penetration of renewables on the electric grid and ready availability of real-time data about electricity usage, the electric power grid is becoming a large-scale complex Cyber-Physical System (CPS) to meet consumer demands for electricity through the day, every day. Reinforcement learning (RL) algorithms offer these CPS systems an approach to seamlessly integrating distributed energy sources into the legacy electric grid more efficiently, effectively and affordably. It also offers significant potential savings in capital investment cost and labor, and greater resiliency to disruptions in service. This research project develops a framework for model-based online reinforcement learning to address several classes of problems. First, it models control of energy CPS as finite horizon RL problems. Second, instead of focusing on asymptotic convergence, this project focuses on optimal finite time performance. Third, while a simplistic learning algorithm might drive an energy CPS to an unsafe region of operations, thereby risking unwanted consequences, this project develops safe RL algorithms that optimize performance and respect safety constraints. Fourth, this project exploits the physical properties of the energy CPS to avoid the dimensionality problems, often associated with RL problems. Lastly, the project develops sequential algorithms using a "contextual bandits" approach for learning consumer specific parameters and adaptively scheduling to account for consumer usage.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着可再生能源在电网中的渗透率不断提高以及实时用电数据的可用性，电网正在成为一个大型复杂的网络物理系统（CPS），以满足消费者全天的用电需求，每天。强化学习 (RL) 算法为这些 CPS 系统提供了一种更高效、更有效且经济实惠地将分布式能源无缝集成到传统电网的方法。它还可以显着节省资本投资成本和劳动力，并提高对服务中断的恢复能力。该研究项目开发了一个基于模型的在线强化学习框架，以解决几类问题。首先，它将能量 CPS 的控制建模为有限视野 RL 问题。其次，该项目不关注渐近收敛，而是关注最优有限时间性能。第三，虽然简单的学习算法可能会将能源 CPS 驱动到不安全的操作区域，从而冒出现不良后果的风险，但该项目开发了安全的 RL 算法，可以优化性能并尊重安全约束。第四，该项目利用能量 CPS 的物理特性来避免通常与 RL 问题相关的维数问题。最后，该项目使用“上下文强盗”方法开发顺序算法，用于学习消费者特定参数并自适应调度以考虑消费者的使用情况。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的评估进行评估，被认为值得支持。影响审查标准。