# PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control

arXiv: https://arxiv.org/abs/2011.12354 · HTML: https://mlsanity.com/paper/2011.12354/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2020.

## Built on by (most cited first, 13 of 13)

- [Hindsight Learning for MDPs with Exogenous Inputs](https://mlsanity.com/paper/2207.06272/) (2022)
- [PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems](https://mlsanity.com/paper/2111.05969/) (2021)
- [Efficient Planning in Combinatorial Action Spaces with Applications to Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.04376/) (2023)
- [CommonPower: A Framework for Safe Data-Driven Smart Grid Control](https://mlsanity.com/paper/2406.03231/) (2024)
- [Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition](https://mlsanity.com/paper/2505.05968/) (2025)
- [A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control](https://mlsanity.com/paper/2312.04371/) (2023)
- [Supporting Future Electrical Utilities: Using Deep Learning Methods in EMS and DMS Algorithms](https://mlsanity.com/paper/2303.00428/) (2023)
- [Decentralized Diffusion Policy Learning for Enhanced Exploration in Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2605.07101/) (2026)
- [Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamics](https://mlsanity.com/paper/2605.30461/) (2026)
- [Leveraging the Potential of Novel Data in Power Line Communication of Electricity Grids](https://mlsanity.com/paper/2209.12693/) (2022)
- [Recent Progress in Energy Management of Connected Hybrid Electric Vehicles Using Reinforcement Learning](https://mlsanity.com/paper/2308.14602/) (2023)
- [Constraint-Guided Prediction Refinement via Deterministic Diffusion Trajectories](https://mlsanity.com/paper/2506.12911/) (2025)
- [Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling](https://mlsanity.com/paper/2605.12411/) (2026)

## Abstract (arXiv)

This paper develops an efficient multi-agent deep reinforcement learning algorithm for cooperative controls in powergrids. Specifically, we consider the decentralized inverter-based secondary voltage control problem in distributed generators (DGs), which is first formulated as a cooperative multi-agent reinforcement learning (MARL) problem. We then propose a novel on-policy MARL algorithm, PowerNet, in which each agent (DG) learns a control policy based on (sub-)global reward but local states from its neighboring agents. Motivated by the fact that a local control from one agent has limited impact on agents distant from it, we exploit a novel spatial discount factor to reduce the effect from remote agents, to expedite the training process and improve scalability. Furthermore, a differentiable, learning-based communication protocol is employed to foster the collaborations among neighboring agents. In addition, to mitigate the effects of system uncertainty and random noise introduced during on-policy learning, we utilize an action smoothing factor to stabilize the policy execution. To facilitate training and evaluation, we develop PGSim, an efficient, high-fidelity powergrid simulation platform. Experimental results in two microgrid setups show that the developed PowerNet outperforms a conventional model-based control, as well as several state-of-the-art MARL algorithms. The decentralized learning scheme and high sample efficiency also make it viable to large-scale power grids.