# Optimal Planning of Hybrid Energy Storage Systems using Curtailed Renewable Energy through Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2212.05662 · HTML: https://mlsanity.com/paper/2212.05662/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 42 references that are in the map.
- First posted on arXiv 12 Dec 2022.

## Builds on (most cited first, 4 of 4)

- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Lifelong Control of Off-grid Microgrid with Model Based Reinforcement Learning](https://mlsanity.com/paper/2005.08006/) (2020)
- [High-Level Penetration of Renewable Energy with Grid: Challenges and Opportunities](https://mlsanity.com/paper/2006.04638/) (2020)

## Abstract (arXiv)

Energy management systems (EMS) are becoming increasingly important in order to utilize the continuously growing curtailed renewable energy. Promising energy storage systems (ESS), such as batteries and green hydrogen should be employed to maximize the efficiency of energy stakeholders. However, optimal decision-making, i.e., planning the leveraging between different strategies, is confronted with the complexity and uncertainties of large-scale problems. Here, we propose a sophisticated deep reinforcement learning (DRL) methodology with a policy-based algorithm to realize the real-time optimal ESS planning under the curtailed renewable energy uncertainty. A quantitative performance comparison proved that the DRL agent outperforms the scenario-based stochastic optimization (SO) algorithm, even with a wide action and observation space. Owing to the uncertainty rejection capability of the DRL, we could confirm a robust performance, under a large uncertainty of the curtailed renewable energy, with a maximizing net profit and stable system. Action-mapping was performed for visually assessing the action taken by the DRL agent according to the state. The corresponding results confirmed that the DRL agent learns the way like what a human expert would do, suggesting reliable application of the proposed methodology.