# Applications of Reinforcement Learning in Deregulated Power Market: A Comprehensive Review

arXiv: https://arxiv.org/abs/2205.08369 · HTML: https://mlsanity.com/paper/2205.08369/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 160 references that are in the map.
- First posted on arXiv 7 May 2022.

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Conservative Offline Distributional Reinforcement Learning](https://mlsanity.com/paper/2107.06106/) (2021)
- [Local Differential Privacy based Federated Learning for Internet of Things](https://mlsanity.com/paper/2004.08856/) (2020)
- [Understanding Neural Networks via Feature Visualization: A survey](https://mlsanity.com/paper/1904.08939/) (2019)
- [Learning to run a power network challenge for training topology controllers](https://mlsanity.com/paper/1912.04211/) (2019)
- [Improving Robustness via Risk Averse Distributional Reinforcement Learning](https://mlsanity.com/paper/2005.00585/) (2020)
- [Appropriate Learning Rates of Adaptive Learning Rate Optimization Algorithms for Training Deep Neural Networks](https://mlsanity.com/paper/2002.09647/) (2020)
- [Efficient and Robust Reinforcement Learning with Uncertainty-based Value Expansion](https://mlsanity.com/paper/1912.05328/) (2019)
- [Dynamic Energy Dispatch Based on Deep Reinforcement Learning in IoT-Driven Smart Isolated Microgrids](https://mlsanity.com/paper/2002.02581/) (2020)
- [Option Hedging with Risk Averse Reinforcement Learning](https://mlsanity.com/paper/2010.12245/) (2020)
- [Monte Carlo Sampling Methods for Approximating Interactive POMDPs](https://mlsanity.com/paper/1401.3455/) (2014)
- [Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning](https://mlsanity.com/paper/2106.06061/) (2021)
- [Optimal Demand Response Using Device Based Reinforcement Learning](https://mlsanity.com/paper/1401.1549/) (2014)
- [Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets](https://mlsanity.com/paper/2107.08083/) (2021)
- [A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch](https://mlsanity.com/paper/1908.11543/) (2019)

## Abstract (arXiv)

The increasing penetration of renewable generations, along with the deregulation and marketization of power industry, promotes the transformation of power market operation paradigms. The optimal bidding strategy and dispatching methodology under these new paradigms are prioritized concerns for both market participants and power system operators, with obstacles of uncertain characteristics, computational efficiency, as well as requirements of hyperopic decision-making. To tackle these problems, the Reinforcement Learning (RL), as an emerging machine learning technique with advantages compared with conventional optimization tools, is playing an increasingly significant role in both academia and industry. This paper presents a comprehensive review of RL applications in deregulated power market operation including bidding and dispatching strategy optimization, based on more than 150 carefully selected literatures. For each application, apart from a paradigmatic summary of generalized methodology, in-depth discussions of applicability and obstacles while deploying RL techniques are also provided. Finally, some RL techniques that have great potentiality to be deployed in bidding and dispatching problems are recommended and discussed.