CAREER: Resilient and Efficient Automatic Control in Energy Infrastructure: An Expert-Guided Policy Optimization Framework

职业：能源基础设施中的弹性和高效自动控制：专家指导的政策优化框架

• 批准号: 2338559
• 负责人: Chaoyue Zhao
• 金额: $ 50.85万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-02-01 至 2029-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2338559&HistoricalAwards=false
• 关键词: CAREER; Resilient; Efficient; Automatic; Control

This Faculty Early Career Development (CAREER) award supports research that will leverage cutting-edge artificial intelligence technologies to significantly enhance the resilience and efficiency of automated control systems within a broad class of energy infrastructure systems. This initiative is crucial as it addresses several substantial limitations faced by existing learning-based decision-making frameworks used in practice. This research will bridge these critical knowledge gaps by developing an analytically rigorous and practically implementable framework that integrates reinforcement learning with mathematical optimization, along with expert-in-the-loop guidance. The successful application of this research is anticipated to yield improvements in efficiency, stability, and security, empowering the energy infrastructure to respond rapidly and securely to uncertainty and disruptive events. Integration of this research into the curriculum at University of Washington will foster training and learning opportunities in reinforcement learning for both graduate and undergraduate students. Educational and outreach activities are designed to increase awareness and interest among K-12 and college students through diverse initiatives, including an interactive artificial intelligence game training platform, video modules to supplement classroom lessons for local high schools, and research engagement with underrepresented students.This project creatively applies the principles of distributionally robust optimization to policy gradient reinforcement learning methods for improving online policy sample efficiency and maintaining stability. The model’s superior numerical performance stems from its unrestricted policy distribution, rejection-free policy updates, as well as monotonic performance and global convergence guarantee through Wasserstein metric-based policy optimization. The expert-in-the-loop reinforcement learning framework effectively leverages expert demonstrations and feedback to ensure safe system operation, accelerate learning, and enhance algorithm convergence. By modifying the advantage function in "susceptible" situations, the framework guides learning direction and addresses reinforcement learning’s weaknesses with limited samples. This research will answer three key questions: How to effectively utilize expert feedback? How to identify states that require expert intervention? And how to achieve an optimal and stable policy independent of expert input? The innovative mathematical models and algorithms generated by this work will contribute to addressing online decision-making challenges for better operations and management of complex energy infrastructure systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该教师早期职业发展（职业）奖支持将利用尖端人工智能技术的研究，以显着提高广泛的能源基础设施系统中自动化控制系统的弹性和效率。该计划至关重要，因为它解决了实践中使用的现有基于学习的决策框架所面临的几个重大局限性。这项研究将通过开发一个分析性严格且可实现的框架来弥合这些关键的知识差距，该框架将强化学习与数学优化以及专家在循环指导相结合。预计这项研究的成功应用将提高效率，稳定性和安全性，从而赋予能源基础设施迅速，安全地对不确定性和破坏性事件的响应。将这项研究整合到华盛顿大学的课程中，将为研究生和本科生增强学习机会。教育和宣传活动旨在通过潜水倡议来提高K-12和大学生之间的意识和兴趣，包括互动的人工智能游戏培训平台，视频模块，以补充当地高中的课堂课程，以及与代表性不足的学生进行研究的参与。该项目在创造性优化的阶级优化的原则上，以提高在线稳定性，以提高在线效率的努力，以确保努力稳定效率。该模型的出色数值绩效源于其不受限制的政策分布，无拒绝的政策更新以及单调性能和通过基于Wasserstein Metric的策略优化的单调性能和全球收敛保证。循环的专家增强学习框架有效地利用了专家演示和反馈，以确保系统操作，加速学习并增强算法融合。通过在“易感”情况下修改优势函数，该框架指导了学习方向，并以有限的样本来解决强化学习的弱点。这项研究将回答三个关键问题：如何有效利用专家反馈？如何确定需要专家干预的国家？以及如何实现独立于专家投入的最佳稳定政策？这项工作生成的创新数学模型和算法将有助于解决在线决策挑战，以解决更好的运营和管理复杂的能源基础设施系统的管理。这项奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的影响审查标准来通过评估来获得的支持。