# Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control

arXiv: https://arxiv.org/abs/2502.08681 · HTML: https://mlsanity.com/paper/2502.08681/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [RLlib: Abstractions for Distributed Reinforcement Learning](https://mlsanity.com/paper/1712.09381/) (2017)
- [Near-optimal Reinforcement Learning in Factored MDPs](https://mlsanity.com/paper/1403.3741/) (2014)
- [Power Grid Congestion Management via Topology Optimization with AlphaZero](https://mlsanity.com/paper/2211.05612/) (2022)
- [Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.05077/) (2017)
- [PowRL: A Reinforcement Learning Framework for Robust Management of Power Networks](https://mlsanity.com/paper/2212.02397/) (2022)
- [Learning to run a power network with trust](https://mlsanity.com/paper/2110.12908/) (2021)
- [Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents](https://mlsanity.com/paper/2304.00765/) (2023)
- [AI-Based Autonomous Line Flow Control via Topology Adjustment for Maximizing Time-Series ATCs](https://mlsanity.com/paper/1911.04263/) (2019)
- [Hierarchical Reinforcement Learning for Power Network Topology Control](https://mlsanity.com/paper/2311.02129/) (2023)
- [HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach](https://mlsanity.com/paper/2405.00629/) (2024)
- [Reinforcement learning for Energies of the future and carbon neutrality: a Challenge Design](https://mlsanity.com/paper/2207.10330/) (2022)
- [Imitation Learning for Intra-Day Power Grid Operation through Topology Actions](https://mlsanity.com/paper/2407.19865/) (2024)
- [State and Action Factorization in Power Grids](https://mlsanity.com/paper/2409.04467/) (2024)
- [Optimal Transmission Topology for Facilitating the Growth of Renewable Power Generation](https://mlsanity.com/paper/2103.15677/) (2021)

## Built on by (most cited first, 5 of 5)

- [Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach](https://mlsanity.com/paper/2503.15190/) (2025)
- [Power Grid Control with Graph-Based Distributed Reinforcement Learning](https://mlsanity.com/paper/2509.02861/) (2025)
- [Learning to Run Power Networks: Effective AlphaZero-inspired Topological Control](https://mlsanity.com/paper/2608.14114/) (2026)
- [Harnessing Implicit Cooperation: A Multi-Agent Reinforcement Learning Approach Towards Decentralized Local Energy Markets](https://mlsanity.com/paper/2602.16062/) (2026)
- [Urban Power Grid Topology and Hierarchy Identification from Open Data](https://mlsanity.com/paper/2606.21352/) (2026)

## Abstract (arXiv)

Power grid operation is becoming more complex due to the increase in generation of renewable energy. The recent series of Learning To Run a Power Network (L2RPN) competitions have encouraged the use of artificial agents to assist human dispatchers in operating power grids. However, the combinatorial nature of the action space poses a challenge to both conventional optimizers and learned controllers. Action space factorization, which breaks down decision-making into smaller sub-tasks, is one approach to tackle the curse of dimensionality. In this study, we propose a centrally coordinated multi-agent (CCMA) architecture for action space factorization. In this approach, regional agents propose actions and subsequently a coordinating agent selects the final action. We investigate several implementations of the CCMA architecture, and benchmark in different experimental settings against various L2RPN baseline approaches. The CCMA architecture exhibits higher sample efficiency and superior final performance than the baseline approaches. The results suggest high potential of the CCMA approach for further application in higher-dimensional L2RPN as well as real-world power grid settings.