# Online Multi-agent Reinforcement Learning for Decentralized Inverter-based Volt-VAR Control

arXiv: https://arxiv.org/abs/2006.12841 · HTML: https://mlsanity.com/paper/2006.12841/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2020.

## Built on by (most cited first, 17 of 17)

- [Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks](https://mlsanity.com/paper/2110.14300/) (2021)
- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem](https://mlsanity.com/paper/2305.17198/) (2023)
- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [RL2: Reinforce Large Language Model to Assist Safe Reinforcement Learning for Energy Management of Active Distribution Networks](https://mlsanity.com/paper/2412.01303/) (2024)
- [Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer](https://mlsanity.com/paper/2206.03721/) (2022)
- [Bi-level Off-policy Reinforcement Learning for Volt/VAR Control Involving Continuous and Discrete Devices](https://mlsanity.com/paper/2104.05902/) (2021)
- [Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning](https://mlsanity.com/paper/2203.12048/) (2022)
- [A Scalable Network-Aware Multi-Agent Reinforcement Learning Framework for Decentralized Inverter-based Voltage Control](https://mlsanity.com/paper/2312.04371/) (2023)
- [Multi-Agent Reinforcement Learning for Energy Networks: Computational Challenges, Progress and Open Problems](https://mlsanity.com/paper/2404.15583/) (2024)
- [Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Control](https://mlsanity.com/paper/2405.08443/) (2024)
- [Reducing Learning Difficulties: One-Step Two-Critic Deep Reinforcement Learning for Inverter-based Volt-Var Control](https://mlsanity.com/paper/2203.16289/) (2022)
- [Supporting Future Electrical Utilities: Using Deep Learning Methods in EMS and DMS Algorithms](https://mlsanity.com/paper/2303.00428/) (2023)
- [Robust Deep Reinforcement Learning for Inverter-based Volt-Var Control in Partially Observable Distribution Networks](https://mlsanity.com/paper/2408.06776/) (2024)
- [Two-Stage Active Distribution Network Voltage Control via LLM-RL Collaboration: A Hybrid Knowledge-Data-Driven Approach](https://mlsanity.com/paper/2602.21715/) (2026)
- [Reducing Action Space: Reference-Model-Assisted Deep Reinforcement Learning for Inverter-based Volt-Var Control](https://mlsanity.com/paper/2210.07360/) (2022)
- [A Human-AI Teaming Framework for Deep Reinforcement Learning-Based Voltage Regulation in Distribution Networks](https://mlsanity.com/paper/2607.19667/) (2026)

## Abstract (arXiv)

The distributed Volt/Var control (VVC) methods have been widely studied for active distribution networks(ADNs), which is based on perfect model and real-time P2P communication. However, the model is always incomplete with significant parameter errors and such P2P communication system is hard to maintain. In this paper, we propose an online multi-agent reinforcement learning and decentralized control framework (OLDC) for VVC. In this framework, the VVC problem is formulated as a constrained Markov game and we propose a novel multi-agent constrained soft actor-critic (MACSAC) reinforcement learning algorithm. MACSAC is used to train the control agents online, so the accurate ADN model is no longer needed. Then, the trained agents can realize decentralized optimal control using local measurements without real-time P2P communication. The OLDC with MACSAC has shown extraordinary flexibility, efficiency and robustness to various computing and communication conditions. Numerical simulations on IEEE test cases not only demonstrate that the proposed MACSAC outperforms the state-of-art learning algorithms, but also support the superiority of our OLDC framework in the online application.