Collaborative Research: Data-driven Power Systems Control with Stability Guarantees

合作研究：数据驱动的电力系统控制与稳定性保证

• 批准号: 2154171
• 负责人: Guannan Qu
• 金额: $ 20万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-03-15 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2154171&HistoricalAwards=false
• 关键词: Collaborative; Research; Data; driven; Power

This NSF project aims to design a new data-driven power system control framework with stability guarantee. Power systems are experiencing a period of rapid changes due to the proliferation of renewable generation and distributed energy resources including solar, electric vehicles, and batteries. Many of these new technologies are interfaced with the grid through power electronic interfaces (i.e., inverters) that can be controlled at a much faster timescale compared to conventional machines. However, how to leverage such flexibility is nontrivial due to the nonlinearity, complexity, and uncertainty in the underlying power network. This project will bring transformative changes by developing new reinforcement learning (RL) algorithms for inverter-based frequency and voltage control with formal stability guarantees. The intellectual merits of the project include (i) a novel framework that bridges Lyapunov control theory and RL, therefore providing stability guarantee for learning-based controllers; (ii) neural network structure design that ensures stability constraint is met by design. The broader impacts of the project include various of new courses development and research opportunities for students interested in both energy systems and machine learning/AI.The proposed research consists of three thrusts. Thrust 1 focuses on developing the algorithmic framework that integrates RL with Lyapunov stability constraints, which serves as a foundation to later thrusts. Specifically, we will leverage analytical models to construct Lyapunov functions and engineer the structure of neural network-based controllers to meet the stability constraints. Thrust 2 uses machine learning to discover new Lyapunov functions for realistic power system models and design stable control policies. Thrust 3 integrates the theory and algorithms developed in Thrusts 1 and 2, and robustifies the controllers against modeling error, and network topology re-configurations in both transmission and distribution grids. The contributions of the project are two-folded. On the theoretical side, the proposed research bridges classic control and learning, where control theory provides the structural constraints that guarantee a controller is stable, and RL with neural networks searches over the large parametric spaces to find the best performing controllers that have this structure. On the practical side, our approach clears a critical hurdle in applying RL to power systems by guaranteeing the stability of the learned policy. We envision our framework will serve as the basis for future learning-based smart power system control architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该NSF项目旨在设计一种新的数据驱动的、具有稳定性保证的电力系统控制框架。由于可再生能源发电和分布式能源（包括太阳能、电动汽车和电池）的激增，电力系统正在经历一个快速变化的时期。许多新技术通过电力电子接口（即逆变器）与电网连接，与传统机器相比，这些新技术可以以更快的时间尺度进行控制。然而，由于底层电力网络的非线性、复杂性和不确定性，如何利用这种灵活性并非易事。该项目将通过开发新的强化学习 (RL) 算法来实现基于逆变器的频率和电压控制，并提供正式的稳定性保证，从而带来变革。该项目的智力优势包括（i）一个连接Lyapunov控制理论和RL的新颖框架，从而为基于学习的控制器提供稳定性保证； (ii)神经网络结构设计，确保设计满足稳定性约束。该项目更广泛的影响包括为对能源系统和机器学习/人工智能感兴趣的学生提供各种新课程开发和研究机会。拟议的研究包括三个主旨。 Thrust 1 专注于开发将 RL 与 Lyapunov 稳定性约束相结合的算法框架，作为后续 Thrust 的基础。具体来说，我们将利用分析模型构建李亚普诺夫函数并设计基于神经网络的控制器的结构以满足稳定性约束。 Thrust 2 使用机器学习来发现现实电力系统模型的新李亚普诺夫函数并设计稳定的控制策略。 Thrust 3 集成了 Thrusts 1 和 2 中开发的理论和算法，并针对建模误差以及输电和配电网中的网络拓扑重新配置增强了控制器。该项目的贡献有两个方面。在理论方面，所提出的研究将经典控制和学习联系起来，其中控制理论提供了保证控制器稳定的结构约束，而带有神经网络的强化学习在大型参数空间中进行搜索，以找到具有这种结构的最佳性能控制器。在实践方面，我们的方法通过保证学习策略的稳定性，清除了将强化学习应用于电力系统的关键障碍。我们设想我们的框架将成为未来基于学习的智能电力系统控制架构的基础。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Stability Constrained Reinforcement Learning for Real-Time Voltage Control

用于实时电压控制的稳定性约束强化学习

• DOI: 10.23919/acc53348.2022.9867476
• 发表时间: 2022-06
• 期刊: 2022 American Control Conference (ACC
• 作者: Shi, Yuanyuan;Qu, Guannan;Low, Steven;Anandkumar, Anima;Wierman, Adam
• 通讯作者: Wierman, Adam