# A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control

arXiv: https://arxiv.org/abs/2312.04371 · HTML: https://mlsanity.com/paper/2312.04371/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 33 references that are in the map.
- First posted on arXiv 7 Dec 2023.

## Builds on (most cited first, 7 of 7)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [Can Increasing Input Dimensionality Improve Deep Reinforcement Learning?](https://mlsanity.com/paper/2003.01629/) (2020)
- [Scalable Reinforcement Learning for Multi-Agent Networked Systems](https://mlsanity.com/paper/1912.02906/) (2019)
- [Online Multi-agent Reinforcement Learning for Decentralized Inverter-based Volt-VAR Control](https://mlsanity.com/paper/2006.12841/) (2020)
- [PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control](https://mlsanity.com/paper/2011.12354/) (2020)
- [Secondary Frequency and Voltage Control of Islanded Microgrids via Distributed Averaging](https://mlsanity.com/paper/1504.06784/) (2015)

## Built on by (most cited first, 2 of 2)

- [Reinforcement Learning for Sustainable Energy: A Survey](https://mlsanity.com/paper/2407.18597/) (2024)
- [Transformer-Based Multi-Agent Reinforcement Learning for Networked Systems with Long-Range Interactions](https://mlsanity.com/paper/2511.13103/) (2025)

## Abstract (arXiv)

This paper addresses the challenges associated with decentralized voltage control in power grids due to an increase in distributed generations (DGs). Traditional model-based voltage control methods struggle with the rapid energy fluctuations and uncertainties of these DGs. While multi-agent reinforcement learning (MARL) has shown potential for decentralized secondary control, scalability issues arise when dealing with a large number of DGs. This problem lies in the dominant centralized training and decentralized execution (CTDE) framework, where the critics take global observations and actions. To overcome these challenges, we propose a scalable network-aware (SNA) framework that leverages network structure to truncate the input to the critic's Q-function, thereby improving scalability and reducing communication costs during training. Further, the SNA framework is theoretically grounded with provable approximation guarantee, and it can seamlessly integrate with multiple multi-agent actor-critic algorithms. The proposed SNA framework is successfully demonstrated in a system with 114 DGs, providing a promising solution for decentralized voltage control in increasingly complex power grid systems.