# A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems

arXiv: https://arxiv.org/abs/2109.12073 · HTML: https://mlsanity.com/paper/2109.12073/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 23 of 23)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View](https://mlsanity.com/paper/1909.03211/) (2019)
- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning](https://mlsanity.com/paper/2005.00406/) (2020)
- [Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/2001.09684/) (2020)
- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case](https://mlsanity.com/paper/1910.07421/) (2019)
- [ChainerRL: A Deep Reinforcement Learning Library](https://mlsanity.com/paper/1912.03905/) (2019)
- [Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networks](https://mlsanity.com/paper/2010.05313/) (2020)
- [Transferable Graph Optimizers for ML Compilers](https://mlsanity.com/paper/2010.12438/) (2020)
- [Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks](https://mlsanity.com/paper/2007.02991/) (2020)
- [Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems](https://mlsanity.com/paper/2003.03681/) (2020)
- [Query-based Targeted Action-Space Adversarial Policies on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2011.07114/) (2020)
- [PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems](https://mlsanity.com/paper/2109.03970/) (2021)
- [Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments](https://mlsanity.com/paper/2006.12576/) (2020)
- [Exact Convex Relaxation of Optimal Power Flow in Radial Networks](https://mlsanity.com/paper/1311.7170/) (2013)

## Built on by (most cited first, 5 of 5)

- [Graph Reinforcement Learning for Power Grids: A Comprehensive Survey](https://mlsanity.com/paper/2407.04522/) (2024)
- [Graph Neural Networks on Factor Graphs for Robust, Fast, and Scalable Linear State Estimation with PMUs](https://mlsanity.com/paper/2304.14680/) (2023)
- [Imitation Game: A Model-based and Imitation Learning Deep Reinforcement Learning Hybrid](https://mlsanity.com/paper/2404.01794/) (2024)
- [Graph neural networks for power grid operational risk assessment under evolving grid topology](https://mlsanity.com/paper/2405.07343/) (2024)
- [A Comparative Study of Graph Representations for GNN-Based Power Grid Control in L2RPN](https://mlsanity.com/paper/2609.02538/) (2026)

## Abstract (arXiv)

Volt-var control (VVC) is the problem of operating power distribution systems within healthy regimes by controlling actuators in power systems. Existing works have mostly adopted the conventional routine of representing the power systems (a graph with tree topology) as vectors to train deep reinforcement learning (RL) policies. We propose a framework that combines RL with graph neural networks and study the benefits and limitations of graph-based policy in the VVC setting. Our results show that graph-based policies converge to the same rewards asymptotically however at a slower rate when compared to vector representation counterpart. We conduct further analysis on the impact of both observations and actions: on the observation end, we examine the robustness of graph-based policy on two typical data acquisition errors in power systems, namely sensor communication failure and measurement misalignment. On the action end, we show that actuators have various impacts on the system, thus using a graph representation induced by power systems topology may not be the optimal choice. In the end, we conduct a case study to demonstrate that the choice of readout function architecture and graph augmentation can further improve training performance and robustness.