Collaborative Research: CPS: Medium: Empowering prosumers in electricity markets through market design and learning

合作研究：CPS：中：通过市场设计和学习为电力市场中的产消者赋权

• 批准号: 2038416
• 负责人: Vijay Subramanian
• 金额: $ 30万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2038416&HistoricalAwards=false
• 关键词: Collaborative; Research; CPS; Medium; Empowering

The availability of vast amounts of operational and end-user data in cyber-physical systems implies that paradigm improvements in monitoring and control can be attained via learning by many artificial intelligence agents despite them possessing vastly different abilities. Engaging this heterogeneous agent base in the context of the smart grid requires the use of hierarchical markets, wherein end-users participate in downstream markets collectively through aggregators, who in turn are coordinated by an upstream market. The goal of this project is to conduct a systematic study of such market-mediated learning and control. This project aims at much deeper levels of participation from end-users contributing electricity generation such as rooftop solar, shedding load via demand response, and providing storage capabilities such as electric vehicle batteries, to transform into reliable distributed energy resources (DER) at the level of wholesale markets. A methodological theme is multi-agent reinforcement learning (MARL) by agents that control physical systems via actions at different levels of the hierarchy. Underlying the whole project are well-founded physical models of the transmission and distribution grids, which provide structure to the problem domain and concrete use cases. This project facilitates a deeper level of decarbonization in the electricity sector, and contributes to climate change solutions by engineering a flat, interactive grid architecture that allows significant DERs to provide electricity services to both local and regional grids. Engagement with a grid-level market operator enables the project to address a problem space of immediate relevance to the current electricity grid. The project also includes the development of educational materials on data-analytics and energy systems. Intrinsic to the program are efforts at outreach to involve high-school students via demonstrations and lectures based on the technology developed.The goal of this project is a systematic and principled study of methods for hierarchical market-mediated learning and control, with the electric grid being the primary application domain. Multi-agent reinforcement learning (MARL) runs as a common methodological theme through the project, with strategic agents with varying information structures and concepts of rationality that control physical systems via actions at different levels of the hierarchy. The approach is different from studies on generic MARL algorithms in that attention is focused on well-founded physical models of the transmission and distribution grids, as well as the workings of the power system. The project is organized into three interdependent thrusts, namely, (i) Learning to bid as aggregators in wholesale markets, which studies dynamics of aggregators that provide supply offers and demand bids at the upstream market (wholesale level), while procuring these services from downstream DERs (retail level), (ii) Learning to incentivize retail users to contribute their resources, under which bounded rational agents learn to respond to a population-level distribution of other agents and incentives provided, and (iii) Evaluation and experimentation over a full-scale system emulator by integrating it with reinforcement learning tools. This project provides an architecture for DERs to provide electricity services to both local and regional grids, and hence contributes to developing solutions to climate change. Engagement with an independent system operator enables a focus on grid-specific issues, ensuring the applicability of the solutions to real-world problems. The impact is enhanced by specific minority inclusion activities, courses on computing tailored to broaden participation in the context of data-analytics and energy systems, and outreach to high-school students using demonstrations and lectures based on the project results.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

网络物理系统中大量操作数据和最终用户数据的可用性意味着，尽管许多人工智能代理拥有截然不同的能力，但可以通过许多人工智能代理的学习来实现监视和控制的范式改进。 在智能电网的背景下参与这种异构代理基础需要使用分层市场，其中最终用户通过聚合器共同参与下游市场，而聚合器又由上游市场进行协调。 该项目的目标是对这种市场介导的学习和控制进行系统研究。 该项目旨在让最终用户更深层次地参与，提供屋顶太阳能等发电、通过需求响应减少负载以及提供电动汽车电池等存储能力，以转化为可靠的分布式能源（DER）的批发市场。 方法论主题是多代理强化学习（MARL），代理通过层次结构不同级别的操作来控制物理系统。 整个项目的基础是基础完善的输配电网物理模型，它为问题领域和具体用例提供了结构。 该项目促进了电力行业更深层次的脱碳，并通过设计一个扁平的、交互式的电网架构，允许重要的分布式能源向当地和区域电网提供电力服务，为气候变化解决方案做出贡献。 与电网级市场运营商的合作使该项目能够解决与当前电网直接相关的问题空间。 该项目还包括开发有关数据分析和能源系统的教育材料。 该计划的本质是努力通过基于所开发技术的演示和讲座来吸引高中生参与。该项目的目标是对分层市场介导的学习和控制方法进行系统和原则性的研究，通过电网是主要的应用领域。 多智能体强化学习（MARL）作为整个项目的共同方法主题，具有不同信息结构和理性概念的战略智能体，通过层次结构不同级别的操作来控制物理系统。 该方法与通用 MARL 算法的研究不同，因为它的注意力集中在有充分依据的输配电网物理模型以及电力系统的工作原理上。 该项目分为三个相互依存的主旨，即（i）学习作为批发市场中的聚合商进行投标，研究在上游市场（批发层面）提供供应报价和需求投标的聚合商的动态，同时从下游采购这些服务DER（零售层面），(ii) 学习激励零售用户贡献其资源，在此情况下，有限理性主体学习对其他主体的人口层面分布和所提供的激励做出反应，以及 (iii) 评估和通过将全尺寸系统模拟器与强化学习工具集成来进行实验。 该项目为分布式能源提供了一个向当地和区域电网提供电力服务的架构，从而有助于制定气候变化解决方案。 与独立系统运营商合作可以专注于电网特定问题，确保解决方案对实际问题的适用性。 通过特定的少数族裔包容性活动、为扩大数据分析和能源系统背景下的参与而量身定制的计算课程，以及根据项目结果通过演示和讲座向高中生进行推广，增强了影响力。该奖项反映了 NSF 的法定奖项使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Empirical Policy Evaluation With Supergraphs

用超级图进行实证政策评估

• DOI: 10.1109/jsait.2021.3073257
• 发表时间: 2021-06
• 期刊: IEEE Journal on Selected Areas in Information Theory
• 作者: Vial, Daniel;Subramanian, Vijay
• 通讯作者: Subramanian, Vijay

A Strong Duality Result for Cooperative Decentralized Constrained POMDPs

合作分散约束 POMDP 的强对偶结果

• 发表时间: 2023-12
• 期刊: Proceedings of the IEEE Conference on Decision Control
• 作者: Nouman Khan and Vijay Subramanian

Learning-based Optimal Admission Control in a Single Server Queuing System

单服务器排队系统中基于学习的最优准入控制

• DOI: 10.1109/allerton49937.2022.9929406
• 发表时间: 2022-09
• 期刊: and Computing (Allerton
• 作者: Zhang, Yili;Cohen, Asaf;Subramanian, Vijay G.
• 通讯作者: Subramanian, Vijay G.

Dynamic Games Among Teams with Delayed Intra-Team Information Sharing

团队间动态博弈，团队内信息共享延迟

• DOI: 10.1007/s13235-022-00424-4
• 发表时间: 2022-02
• 期刊: Dynamic Games and Applications
• 影响因子: 1.5
• 作者: Tang, Dengwang;Tavafoghi, Hamidreza;Subramanian, Vijay;Nayyar, Ashutosh;Teneketzis, Demosthenis
• 通讯作者: Teneketzis, Demosthenis

Private Information Compression in Dynamic Games among Teams

团队动态博弈中的私有信息压缩

• DOI: 10.1109/cdc45484.2021.9683479
• 发表时间: 2021-01
• 期刊: 2021 60th IEEE Conference on Decision and Control (CDC
• 作者: Tang, Dengwang;Tavafoghi, Hamidreza;Subramanian, Vijay;Nayyar, Ashutosh;Teneketzis, Demosthenis
• 通讯作者: Teneketzis, Demosthenis