# Power Grid Control with Graph-Based Distributed Reinforcement Learning

arXiv: https://arxiv.org/abs/2509.02861 · HTML: https://mlsanity.com/paper/2509.02861/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [HINormer: Representation Learning On Heterogeneous Information Networks with Graph Transformer](https://mlsanity.com/paper/2302.11329/) (2023)
- [Reinforcement Learning for Electricity Network Operation](https://mlsanity.com/paper/2003.07339/) (2020)
- [Power Grid Congestion Management via Topology Optimization with AlphaZero](https://mlsanity.com/paper/2211.05612/) (2022)
- [PowRL: A Reinforcement Learning Framework for Robust Management of Power Networks](https://mlsanity.com/paper/2212.02397/) (2022)
- [Hierarchical Reinforcement Learning for Power Network Topology Control](https://mlsanity.com/paper/2311.02129/) (2023)
- [HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach](https://mlsanity.com/paper/2405.00629/) (2024)
- [Reinforcement learning for Energies of the future and carbon neutrality: a Challenge Design](https://mlsanity.com/paper/2207.10330/) (2022)
- [Graph Reinforcement Learning for Power Grids: A Comprehensive Survey](https://mlsanity.com/paper/2407.04522/) (2024)
- [Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control](https://mlsanity.com/paper/2502.08681/) (2025)
- [Optimizing Power Grid Topologies with Reinforcement Learning: A Survey of Methods and Challenges](https://mlsanity.com/paper/2504.08210/) (2025)
- [Graph Neural Networks for Transmission Grid Topology Control: Busbar Information Asymmetry and Heterogeneous Representations](https://mlsanity.com/paper/2501.07186/) (2025)

## Built on by (most cited first, 1 of 1)

- [Data-Assimilation-Assisted Reinforcement Learning for Power Grid Control under Load Uncertainty](https://mlsanity.com/paper/2609.27229/) (2026)

## Abstract (arXiv)

The necessary integration of renewable energy sources, combined with the expanding scale of power networks, presents significant challenges in controlling modern power grids. Traditional control systems, which are human and optimization-based, struggle to adapt and to scale in such an evolving context, motivating the exploration of more dynamic and distributed control strategies. This work advances a graph-based distributed reinforcement learning framework for real-time, scalable grid management. The proposed architecture consists of a network of distributed low-level agents acting on individual power lines and coordinated by a high-level manager agent. A Graph Neural Network (GNN) is employed to encode the network's topological information within the single low-level agent's observation. To accelerate convergence and enhance learning stability, the framework integrates imitation learning and potential-based reward shaping. In contrast to conventional decentralized approaches that decompose only the action space while relying on global observations, this method also decomposes the observation space. Each low-level agent acts based on a structured and informative local view of the environment constructed through the GNN. Experiments on the Grid2Op simulation environment show the effectiveness of the approach, which consistently outperforms the standard baseline commonly adopted in the field. Additionally, the proposed model proves to be much more computationally efficient than the simulation-based Expert method.