| 引用本文: | 黄帅博,陈蓓,高降宇.基于马尔可夫决策过程的电动汽车充电站能量管理策略[J].电力自动化设备,2022,42(10): |
| HUANG Shuaibo,CHEN Bei,GAO Jiangyu.Energy management strategy of electric vehicle charging station based on Markov decision process[J].Electric Power Automation Equipment,2022,42(10): |
|
| 摘要: |
| 电动汽车充电站作为并网分布式储能装置，是实现电动汽车与未来能源互联网深度融合的重要组成部分。考虑分时电价和电动汽车用户行为的不确定性，提出了以电动汽车充电站日运营成本最小化为目标的能量管理策略。为了减少对先验信息的依赖和约束，将优化问题建模为一个新的有限回合马尔可夫决策过程模型；基于传统成本模型提出奖惩回报函数，通过主动学习调度决策，得到每辆电动汽车的实时充放电行为；针对模型的高维状态空间问题，设计相应的状态空间和动作空间，采用一种卷积神经网络结构结合强化学习的方法，通过从原始数据观测中提取高质量的经验，获取最优调度策略以达到优化目标。仿真结果表明，与传统的充电策略相比，所提策略可以有效地降低充电站的日运营成本，保护电动汽车的电池，同时能满足电动汽车用户的充电需求。 |
| 关键词: 电动汽车 充电站 充电规划 马尔可夫决策过程 能量管理 深度强化学习 |
| DOI：10.16081/j.epae.202208031 |
| 分类号:U469.72;TM73 |
| 基金项目:国家自然科学基金资助项目(62173222,62073139)；国家科技攻关计划重大项目(2020AAA0109301) |
|
| Energy management strategy of electric vehicle charging station based on Markov decision process |
|
HUANG Shuaibo, CHEN Bei, GAO Jiangyu
|
|
School of Electronic and Electrical Engineering, Shanghai University of Engineering and Technology, Shanghai 201620, China
|
| Abstract: |
| As the grid-connected distributed energy storage devices, the EVCSs(Electric Vehicle Charging Stations) are the important part of realizing the deep integration of EVs(Electric Vehicles) and the future energy Internet. Considering the time-of-use electricity price and the uncertainties of EV users’ behavior, an energy management strategy is proposed to minimize the daily EVCS operation cost. In order to reduce the dependence and constraint on prior information, the optimization problem is modeled as a new finite round MDP(Markov Decision Process) model. Based on the traditional cost model, the reward and punishment return function is proposed, and the real-time charging and discharging behavior of each EV is obtained by actively learning the scheduling decision. Aiming at the high-dimensional state space problem of the model, the corresponding state space and action space are designed. A method of combining the convolution neural network structure with reinforcement learning is adopted to extract high-quality experience from the original observation data and obtain the optimal scheduling strategy to achieve the optimization objective. Simulative results show that compared with the traditional charging strategy, the proposed strategy can effectively reduce the daily EVCS operation cost, protect the battery of EVs, and meet the charging demands of EV users. |
| Key words: electric vehicles charging station charging planning Markov decision process energy management deep reinforcement learning |
