# PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems

arXiv: https://arxiv.org/abs/2109.03970 · HTML: https://mlsanity.com/paper/2109.03970/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation](https://mlsanity.com/paper/2002.09516/) (2020)
- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [ChainerRL: A Deep Reinforcement Learning Library](https://mlsanity.com/paper/1912.03905/) (2019)
- [Learning to run a power network challenge for training topology controllers](https://mlsanity.com/paper/1912.04211/) (2019)
- [Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systems](https://mlsanity.com/paper/2103.07932/) (2021)
- [Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems](https://mlsanity.com/paper/2003.03681/) (2020)
- [Deep Reinforcement Learning](https://mlsanity.com/paper/1810.06339/) (2018)
- [pymgrid: An Open-Source Python Microgrid Simulator for Applied Artificial Intelligence Research](https://mlsanity.com/paper/2011.08004/) (2020)
- [Continuous Multiagent Control using Collective Behavior Entropy for Large-Scale Home Energy Management](https://mlsanity.com/paper/2005.10000/) (2020)
- [ns3-gym: Extending OpenAI Gym for Networking Research](https://mlsanity.com/paper/1810.03943/) (2018)
- [Exact Convex Relaxation of Optimal Power Flow in Radial Networks](https://mlsanity.com/paper/1311.7170/) (2013)

## Built on by (most cited first, 14 of 14)

- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Graph Reinforcement Learning for Power Grids: A Comprehensive Survey](https://mlsanity.com/paper/2407.04522/) (2024)
- [A Graph Policy Network Approach for Volt-Var Control in Power Distribution Systems](https://mlsanity.com/paper/2109.12073/) (2021)
- [CommonPower: A Framework for Safe Data-Driven Smart Grid Control](https://mlsanity.com/paper/2406.03231/) (2024)
- [Soft Actor-Critic With Integer Actions](https://mlsanity.com/paper/2109.08512/) (2021)
- [Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Control](https://mlsanity.com/paper/2207.13730/) (2022)
- [HistoGym: A Reinforcement Learning Environment for Histopathological Image Analysis](https://mlsanity.com/paper/2408.08847/) (2024)
- [Scalable Volt-VAR Optimization using RLlib-IMPALA Framework: A Reinforcement Learning Approach](https://mlsanity.com/paper/2402.15932/) (2024)
- [A Reinforcement Learning Engine with Reduced Action and State Space for Scalable Cyber-Physical Optimal Response](https://mlsanity.com/paper/2410.04518/) (2024)
- [Digital Twin-Empowered Voltage Control for Power Systems](https://mlsanity.com/paper/2412.06940/) (2024)
- [Utilizing Adversarial Training for Robust Voltage Control: An Adaptive Deep Reinforcement Learning Method](https://mlsanity.com/paper/2603.23648/) (2026)
- [OpenG2G: A Simulation Platform for AI Datacenter-Grid Runtime Coordination](https://mlsanity.com/paper/2605.05519/) (2026)
- [PowerZooJax: A JAX-based Power System Benchmark for Reinforcement Learning](https://mlsanity.com/paper/2609.36052/) (2026)
- [Training Numerical Intelligence via Auto-Diagnosis and Skill Discovery](https://mlsanity.com/paper/2610.03872/) (2026)

## Abstract (arXiv)

We introduce PowerGym, an open-source reinforcement learning environment for Volt-Var control in power distribution systems. Following OpenAI Gym APIs, PowerGym targets minimizing power loss and voltage violations under physical networked constraints. PowerGym provides four distribution systems (13Bus, 34Bus, 123Bus, and 8500Node) based on IEEE benchmark systems and design variants for various control difficulties. To foster generalization, PowerGym offers a detailed customization guide for users working with their distribution systems. As a demonstration, we examine state-of-the-art reinforcement learning algorithms in PowerGym and validate the environment by studying controller behaviors. The repository is available at urlhttps://github.com/siemens/powergym.