| 引用本文: | 陈亭轩,徐潇源,严正,朱彦名.基于深度强化学习的光储充电站储能系统优化运行[J].电力自动化设备,2021,41(10): |
| CHEN Tingxuan,XU Xiaoyuan,YAN Zheng,ZHU Yanming.Optimal operation based on deep reinforcement learning for energy storage system in photovoltaic-storage charging station[J].Electric Power Automation Equipment,2021,41(10): |
|
| 摘要: |
| 优化储能充放电策略有利于提升光储充电站运行经济性，但是现有模型驱动的随机优化方法无法全面考虑储能系统的复杂运行特性以及光伏发电功率、电动汽车充电负荷的不确定性。因此，提出一种基于深度强化学习的光储充电站储能系统全寿命周期优化运行方法。首先对储能运行效率模型和容量衰减模型进行精细化建模。然后考虑电动汽车充电需求、光伏出力和电价的不确定性，在满足电动汽车充电需求和光伏消纳的条件下，以光储充电站收益最大化为目标，建立了基于强化学习的储能优化运行问题。考虑到储能充放电决策动作的连续性，采用双延迟深度确定性策略梯度算法进行求解。采用实际历史数据对模型进行训练，根据当前时段状态对储能充放电策略进行实时优化。最后，对所提方法及模型进行测试，并将所提出的方法与传统模型驱动方法进行对比，结果验证了所提方法及模型的有效性。 |
| 关键词: 储能 光储充电站 不确定性 深度强化学习 优化 |
| DOI：10.16081/j.epae.202110037 |
| 分类号:TM715;U469.72 |
| 基金项目:国家自然科学基金资助项目(52077136) |
|
| Optimal operation based on deep reinforcement learning for energy storage system in photovoltaic-storage charging station |
|
CHEN Tingxuan, XU Xiaoyuan, YAN Zheng, ZHU Yanming
|
|
Key Laboratory of Control of Power Transmission and Conversion, Ministry of Education, Shanghai Jiao Tong University, Shanghai 200240, China
|
| Abstract: |
| Optimizing the energy storage charging and discharging strategy of photovoltaic-storage charging stations is conducive to improving the economics of system operation, but the existing model-driven stochastic optimization methods cannot fully consider the accurate energy storage system operating characteristics and the uncertainty of photovoltaic power generation and electric vehicle charging load. In this regard, an optimal operation method based on deep reinforcement learning for the entire life cycle of energy storage system in photovoltaic-storage charging station is proposed. Firstly, the refined model of energy storage operation efficiency and capacity degradation are modeled. Then considering the uncertainty of electric vehicle charging demand, photovoltaic output and electricity price, under the condition of meeting electric vehicle charging demand and photovoltaic consumption, an optimal operation method for energy storage based on reinforcement learning is established, which takes maximizing the total revenue of photovoltaic-storage charging station as its target. Considering the action continuity of the energy storage charging and discharging decision-making, the twin delayed deep deterministic strategy gradient algorithm is used to solve the problem. The historical data is used to train the model, and the energy storage charging and discharging strategy can be optimized in real time according to the current state. Finally, the proposed method and model are tested and compared with the traditional model-driven methods, the results verify the effectiveness of the proposed method and model. |
| Key words: energy storage photovoltaic-storage charging station uncertainty deep reinforcement learning optimization |
