A Modified Rainbow-Based Deep Reinforcement Learning Method for Optimal Scheduling of Charging Station

To improve the operating efficiency and economic benefits, this article proposes a modified rainbow-based deep reinforcement learning (DRL) strategy to realize the charging station (CS) optimal scheduling. As the charging process is a real-time matching between electric vehicles '(EVs) charging demand and CS equipment resources, the CS charging scheduling problem is duly formulated as a finite Markov decision process (FMDP). Considering the multi-stakeholder interaction among EVs, CSs, and distribution networks (DNs), a comprehensive information perception model was constructed to extract the environmental state required by the agent. According to the random behavior characteristics of the EV charging arrival and departure times, the startup of the charging pile control module was regarded as the agent's action space. To tackle this issue, the modified rainbow approach was utilized to develop a time-scale-based CS scheme to compensate for the resource requirements mismatch on the energy scale. Case studies were conducted within a CS integrated with the photovoltaic and energy storage system. The results reveal that the proposed method effectively reduces the CS operating cost and improves the new energy consumption.

为提高运营效率和经济效益，本文提出一种基于改进彩虹算法的深度强化学习（DRL）策略，以实现充电站（CS）的最优调度。由于充电过程是电动汽车（EV）充电需求与充电站设备资源之间的实时匹配，充电站充电调度问题被恰当地表述为有限马尔可夫决策过程（FMDP）。考虑到电动汽车、充电站和配电网（DN）之间的多利益相关者相互作用，构建了一个综合信息感知模型以提取智能体所需的环境状态。根据电动汽车充电到达和离开时间的随机行为特征，将充电桩控制模块的启动视为智能体的动作空间。为解决这一问题，利用改进的彩虹算法开发了一种基于时间尺度的充电站方案，以弥补能量尺度上的资源需求不匹配。在一个集成了光伏和储能系统的充电站中进行了案例研究。结果表明，所提出的方法有效地降低了充电站的运营成本，并提高了新能源的消耗。