CAREER: Resilient and Efficient Automatic Control in Energy Infrastructure: An Expert-Guided Policy Optimization Framework

职业：能源基础设施中的弹性和高效自动控制：专家指导的政策优化框架

• 批准号: 2338559
• 负责人: Chaoyue Zhao
• 金额: $ 50.85万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-02-01 至 2029-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2338559&HistoricalAwards=false
• 关键词: CAREER; Resilient; Efficient; Automatic; Control

This Faculty Early Career Development (CAREER) award supports research that will leverage cutting-edge artificial intelligence technologies to significantly enhance the resilience and efficiency of automated control systems within a broad class of energy infrastructure systems. This initiative is crucial as it addresses several substantial limitations faced by existing learning-based decision-making frameworks used in practice. This research will bridge these critical knowledge gaps by developing an analytically rigorous and practically implementable framework that integrates reinforcement learning with mathematical optimization, along with expert-in-the-loop guidance. The successful application of this research is anticipated to yield improvements in efficiency, stability, and security, empowering the energy infrastructure to respond rapidly and securely to uncertainty and disruptive events. Integration of this research into the curriculum at University of Washington will foster training and learning opportunities in reinforcement learning for both graduate and undergraduate students. Educational and outreach activities are designed to increase awareness and interest among K-12 and college students through diverse initiatives, including an interactive artificial intelligence game training platform, video modules to supplement classroom lessons for local high schools, and research engagement with underrepresented students.This project creatively applies the principles of distributionally robust optimization to policy gradient reinforcement learning methods for improving online policy sample efficiency and maintaining stability. The model’s superior numerical performance stems from its unrestricted policy distribution, rejection-free policy updates, as well as monotonic performance and global convergence guarantee through Wasserstein metric-based policy optimization. The expert-in-the-loop reinforcement learning framework effectively leverages expert demonstrations and feedback to ensure safe system operation, accelerate learning, and enhance algorithm convergence. By modifying the advantage function in "susceptible" situations, the framework guides learning direction and addresses reinforcement learning’s weaknesses with limited samples. This research will answer three key questions: How to effectively utilize expert feedback? How to identify states that require expert intervention? And how to achieve an optimal and stable policy independent of expert input? The innovative mathematical models and algorithms generated by this work will contribute to addressing online decision-making challenges for better operations and management of complex energy infrastructure systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该教师早期职业发展（职业）奖支持利用尖端人工智能技术显着提高各类能源基础设施系统中自动化控制系统的弹性和效率的研究。这一举措至关重要，因为它解决了一些重大限制。这项研究将通过开发一个分析严谨且可实际实施的框架来弥补这些关键知识差距，该框架将强化学习与数学优化以及专家在环指导相结合。预计该研究的成功应用提高效率、稳定性和安全性，使能源基础设施能够快速、安全地应对不确定性和破坏性事件。将这项研究纳入华盛顿大学的课程将为研究生和本科生提供强化学习的培训和学习机会。教育和外展活动旨在通过各种举措提高 K-12 和大学生的认识和兴趣，包括交互式人工智能游戏培训平台、补充当地高中课堂课程的视频模块以及对代表性不足的学生的研究参与。 .该项目创造性地应用了以下原则对策略梯度强化学习方法进行分布鲁棒优化，以提高在线策略样本效率并保持稳定性。该模型优越的数值性能源于其无限制的策略分布、无拒绝的策略更新，以及通过 Wasserstein 度量的单调性能和全局收敛保证。基于专家在环的强化学习框架有效利用专家论证和反馈，通过修改“敏感”中的优势函数来确保系统安全运行，加速学习并增强算法收敛性。该研究将回答三个关键问题：如何有效利用专家反馈？如何识别需要专家干预的状态？以及如何实现最优且稳定的政策独立。这项工作产生的创新数学模型和算法将有助于解决在线决策挑战，从而更好地运营和管理复杂的能源基础设施系统。该奖项是 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准。