# Heterogeneous Multi-Agent Proximal Policy Optimization for Power Distribution System Restoration

arXiv: https://arxiv.org/abs/2511.14730 · HTML: https://mlsanity.com/paper/2511.14730/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 18 Nov 2025.

## Builds on (most cited first, 6 of 6)

- [Heterogeneous-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.09870/) (2023)
- [Learning Optimal Crew Dispatch for Grid Restoration Following an Earthquake](https://mlsanity.com/paper/2509.04308/) (2025)
- [Critical Load Restoration using Distributed Energy Resources for Resilient Power Distribution System](https://mlsanity.com/paper/1912.04535/) (2019)
- [Curriculum-based Reinforcement Learning for Distribution System Critical Load Restoration](https://mlsanity.com/paper/2203.04166/) (2022)
- [Multi-agent Deep Reinforcement Learning for Distributed Load Restoration](https://mlsanity.com/paper/2306.14018/) (2023)
- [The Role of Distributed Energy Resources in Distribution System Restoration](https://mlsanity.com/paper/2112.06317/) (2021)

## Built on by (most cited first, 1 of 1)

- [Event-Driven Deep RL Dispatcher for Post-Storm Distribution System Restoration](https://mlsanity.com/paper/2601.10044/) (2026)

## Abstract (arXiv)

Restoring power distribution systems (PDSs) after large-scale outages requires sequential switching actions that reconfigure feeder topology and coordinate distributed energy resources (DERs) under nonlinear constraints, including power balance, voltage limits, and thermal ratings. These challenges limit the scalability of conventional optimization and value-based reinforcement learning (RL) approaches. This paper applies a Heterogeneous-Agent Reinforcement Learning (HARL) framework via Heterogeneous-Agent Proximal Policy Optimization (HAPPO) to enable coordinated restoration across interconnected microgrids. Each agent controls a distinct microgrid with different loads, DER capacities, and switch counts. Decentralized actors are trained with a centralized critic for stable on-policy learning, while a physics-informed OpenDSS environment enforces electrical feasibility. Experiments on IEEE 123-bus and 8500-node feeders show HAPPO outperforms PPO, QMIX, Mean-Field RL, and other baselines in restored power, convergence stability, and multi-seed reproducibility. Under a 2400 kW generation cap, the framework restores over 95% of available load on both systems with low-latency execution, supporting practical real-time PDS restoration.