# Performance Comparison of Deep RL Algorithms for Energy Systems Optimal Scheduling

arXiv: https://arxiv.org/abs/2208.00728 · HTML: https://mlsanity.com/paper/2208.00728/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 1 Aug 2022.

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Management](https://mlsanity.com/paper/2012.10504/) (2020)

## Built on by (most cited first, 4 of 4)

- [When Maximum Entropy Misleads Policy Optimization](https://mlsanity.com/paper/2506.05615/) (2025)
- [RL-ADN: A High-Performance Deep Reinforcement Learning Environment for Optimal Energy Storage Systems Dispatch in Active Distribution Networks](https://mlsanity.com/paper/2408.03685/) (2024)
- [A Constraint Enforcement Deep Reinforcement Learning Framework for Optimal Energy Storage Systems Dispatch](https://mlsanity.com/paper/2307.14304/) (2023)
- [Safe Imitation Learning-based Optimal Energy Storage Systems Dispatch in Distribution Networks](https://mlsanity.com/paper/2411.00995/) (2024)

## Abstract (arXiv)

Taking advantage of their data-driven and model-free features, Deep Reinforcement Learning (DRL) algorithms have the potential to deal with the increasing level of uncertainty due to the introduction of renewable-based generation. To deal simultaneously with the energy systems' operational cost and technical constraints (e.g, generation-demand power balance) DRL algorithms must consider a trade-off when designing the reward function. This trade-off introduces extra hyperparameters that impact the DRL algorithms' performance and capability of providing feasible solutions. In this paper, a performance comparison of different DRL algorithms, including DDPG, TD3, SAC, and PPO, are presented. We aim to provide a fair comparison of these DRL algorithms for energy systems optimal scheduling problems. Results show DRL algorithms' capability of providing in real-time good-quality solutions, even in unseen operational scenarios, when compared with a mathematical programming model of the energy system optimal scheduling problem. Nevertheless, in the case of large peak consumption, these algorithms failed to provide feasible solutions, which can impede their practical implementation.