# PowRL: A Reinforcement Learning Framework for Robust Management of Power Networks

arXiv: https://arxiv.org/abs/2212.02397 · HTML: https://mlsanity.com/paper/2212.02397/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 5 Dec 2022.

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [Learning to run a power network challenge for training topology controllers](https://mlsanity.com/paper/1912.04211/) (2019)
- [Action Set Based Policy Optimization for Safe Power Grid Management](https://mlsanity.com/paper/2106.15200/) (2021)
- [AI-Based Autonomous Line Flow Control via Topology Adjustment for Maximizing Time-Series ATCs](https://mlsanity.com/paper/1911.04263/) (2019)
- [An Efficient Asynchronous Method for Integrating Evolutionary and Gradient-based Policy Search](https://mlsanity.com/paper/2012.05417/) (2020)
- [Introducing machine learning for power system operation support](https://mlsanity.com/paper/1709.09527/) (2017)
- [Graph-based Preconditioning Conjugate Gradient Algorithm for N-1 Contingency Analysis](https://mlsanity.com/paper/1803.03290/) (2018)

## Built on by (most cited first, 18 of 18)

- [Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents](https://mlsanity.com/paper/2304.00765/) (2023)
- [Hierarchical Reinforcement Learning for Power Network Topology Control](https://mlsanity.com/paper/2311.02129/) (2023)
- [HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach](https://mlsanity.com/paper/2405.00629/) (2024)
- [RL2Grid: Benchmarking Reinforcement Learning in Power Grid Operations](https://mlsanity.com/paper/2503.23101/) (2025)
- [Graph Reinforcement Learning for Power Grids: A Comprehensive Survey](https://mlsanity.com/paper/2407.04522/) (2024)
- [PFDelta: A Benchmark Dataset for Power Flow under Load, Generation, and Topology Variations](https://mlsanity.com/paper/2510.22048/) (2025)
- [Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control](https://mlsanity.com/paper/2502.08681/) (2025)
- [Blackout Mitigation via Physics-guided RL](https://mlsanity.com/paper/2401.09640/) (2024)
- [State and Action Factorization in Power Grids](https://mlsanity.com/paper/2409.04467/) (2024)
- [Fault Detection for agents on power grid topology optimization: A Comprehensive analysis](https://mlsanity.com/paper/2406.16426/) (2024)
- [Graph Neural Networks for Transmission Grid Topology Control: Busbar Information Asymmetry and Heterogeneous Representations](https://mlsanity.com/paper/2501.07186/) (2025)
- [Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach](https://mlsanity.com/paper/2503.15190/) (2025)
- [Multi-Objective Reinforcement Learning for Power Grid Topology Control](https://mlsanity.com/paper/2502.00040/) (2025)
- [Power Grid Control with Graph-Based Distributed Reinforcement Learning](https://mlsanity.com/paper/2509.02861/) (2025)
- [Physics Informed Reinforcement Learning with Gibbs Priors for Topology Control in Power Grids](https://mlsanity.com/paper/2604.01830/) (2026)
- [Learning to Run Power Networks: Effective AlphaZero-inspired Topological Control](https://mlsanity.com/paper/2608.14114/) (2026)
- [Transferable Graph Learning for Transmission Congestion Management via Busbar Splitting](https://mlsanity.com/paper/2510.20591/) (2025)
- [Data-Assimilation-Assisted Reinforcement Learning for Power Grid Control under Load Uncertainty](https://mlsanity.com/paper/2609.27229/) (2026)

## Abstract (arXiv)

Power grids, across the world, play an important societal and economical role by providing uninterrupted, reliable and transient-free power to several industries, businesses and household consumers. With the advent of renewable power resources and EVs resulting into uncertain generation and highly dynamic load demands, it has become ever so important to ensure robust operation of power networks through suitable management of transient stability issues and localize the events of blackouts. In the light of ever increasing stress on the modern grid infrastructure and the grid operators, this paper presents a reinforcement learning (RL) framework, PowRL, to mitigate the effects of unexpected network events, as well as reliably maintain electricity everywhere on the network at all times. The PowRL leverages a novel heuristic for overload management, along with the RL-guided decision making on optimal topology selection to ensure that the grid is operated safely and reliably (with no overloads). PowRL is benchmarked on a variety of competition datasets hosted by the L2RPN (Learning to Run a Power Network). Even with its reduced action space, PowRL tops the leaderboard in the L2RPN NeurIPS 2020 challenge (Robustness track) at an aggregate level, while also being the top performing agent in the L2RPN WCCI 2020 challenge. Moreover, detailed analysis depicts state-of-the-art performances by the PowRL agent in some of the test scenarios.