# Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems

arXiv: https://arxiv.org/abs/2003.03681 · HTML: https://mlsanity.com/paper/2003.03681/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2020.

## Built on by (most cited first, 17 of 17)

- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems](https://mlsanity.com/paper/2109.03970/) (2021)
- [GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management](https://mlsanity.com/paper/2110.06396/) (2021)
- [Massively Digitized Power Grid: Opportunities and Challenges of Use-inspired AI](https://mlsanity.com/paper/2205.05180/) (2022)
- [A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems](https://mlsanity.com/paper/2109.12073/) (2021)
- [CommonPower: A Framework for Safe Data-Driven Smart Grid Control](https://mlsanity.com/paper/2406.03231/) (2024)
- [Reducing Learning Difficulties: One-Step Two-Critic Deep Reinforcement Learning for Inverter-based Volt-Var Control](https://mlsanity.com/paper/2203.16289/) (2022)
- [Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information](https://mlsanity.com/paper/2401.15848/) (2024)
- [Residual Deep Reinforcement Learning for Inverter-based Volt-Var Control](https://mlsanity.com/paper/2408.06790/) (2024)
- [A Reinforcement Learning-based Volt-VAR Control Dataset and Testing Environment](https://mlsanity.com/paper/2204.09500/) (2022)
- [Reducing Action Space: Reference-Model-Assisted Deep Reinforcement Learning for Inverter-based Volt-Var Control](https://mlsanity.com/paper/2210.07360/) (2022)
- [Distributionally Robust Safety Filter for Learning-Based Control in Active Distribution Systems](https://mlsanity.com/paper/2307.16351/) (2023)
- [Digital Twin-Empowered Voltage Control for Power Systems](https://mlsanity.com/paper/2412.06940/) (2024)
- [Partially Observable Residual Reinforcement Learning for PV-Inverter-Based Voltage Control in Distribution Grids](https://mlsanity.com/paper/2506.19353/) (2025)
- [Utilizing Adversarial Training for Robust Voltage Control: An Adaptive Deep Reinforcement Learning Method](https://mlsanity.com/paper/2603.23648/) (2026)
- [A Human-AI Teaming Framework for Deep Reinforcement Learning-Based Voltage Regulation in Distribution Networks](https://mlsanity.com/paper/2607.19667/) (2026)
- [Safety Screening for Voltage Control in Active Distribution Grids via Distributionally Robust Conformal Screening](https://mlsanity.com/paper/2608.30889/) (2026)

## Abstract (arXiv)

This paper develops a model-free volt-VAR optimization (VVO) algorithm via multi-agent deep reinforcement learning (MADRL) in unbalanced distribution systems. This method is novel since we cast the VVO problem in unbalanced distribution networks to an intelligent deep Q-network (DQN) framework, which avoids solving a specific optimization model directly when facing time-varying operating conditions of the systems. We consider statuses/ratios of switchable capacitors, voltage regulators, and smart inverters installed at distributed generators as the action variables of the DQN agents. A delicately designed reward function guides these agents to interact with the distribution system, in the direction of reinforcing voltage regulation and power loss reduction simultaneously. The forward-backward sweep method for radial three-phase distribution systems provides accurate power flow results within a few iterations to the DQN environment. Finally, the proposed multi-objective MADRL method realizes the dual goals for VVO. We test this algorithm on the unbalanced IEEE 13-bus and 123-bus systems. Numerical simulations validate the excellent performance of this method in voltage regulation and power loss reduction.