# Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systems

arXiv: https://arxiv.org/abs/2103.07932 · HTML: https://mlsanity.com/paper/2103.07932/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 14 Mar 2021.

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [A Rewriting System for Convex Optimization Problems](https://mlsanity.com/paper/1709.04494/) (2017)
- [Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation](https://mlsanity.com/paper/1811.00145/) (2018)
- [Reinforcement Learning and Deep Learning based Lateral Control for Autonomous Driving](https://mlsanity.com/paper/1810.12778/) (2018)
- [Active network management for electrical distribution systems: problem formulation, benchmark, and approximate solution](https://mlsanity.com/paper/1405.2806/) (2014)

## Built on by (most cited first, 18 of 18)

- [Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks](https://mlsanity.com/paper/2110.14300/) (2021)
- [PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems](https://mlsanity.com/paper/2109.03970/) (2021)
- [Learning the Optimal Power Flow: Environment Design Matters](https://mlsanity.com/paper/2403.17831/) (2024)
- [Gym-ANM: Open-source software to leverage reinforcement learning for power system management in research and education](https://mlsanity.com/paper/2105.08846/) (2021)
- [Reinforcement Learning in Practice: Opportunities and Challenges](https://mlsanity.com/paper/2202.11296/) (2022)
- [Improving Policy Optimization via varepsilon-Retrain](https://mlsanity.com/paper/2406.08315/) (2024)
- [Optimizing with Low Budgets: a Comparison on the Black-box Optimization Benchmarking Suite and OpenAI Gym](https://mlsanity.com/paper/2310.00077/) (2023)
- [CommonPower: A Framework for Safe Data-Driven Smart Grid Control](https://mlsanity.com/paper/2406.03231/) (2024)
- [RL-ADN: A High-Performance Deep Reinforcement Learning Environment for Optimal Energy Storage Systems Dispatch in Active Distribution Networks](https://mlsanity.com/paper/2408.03685/) (2024)
- [Reinforcement Learning for Sustainable Energy: A Survey](https://mlsanity.com/paper/2407.18597/) (2024)
- [KKT-Informed Neural Network](https://mlsanity.com/paper/2409.09087/) (2024)
- [Gym-TORAX: Open-source software for integrating reinforcement learning with plasma control simulators in tokamak research](https://mlsanity.com/paper/2510.11283/) (2025)
- [Repairing Reward Functions with Feedback to Mitigate Reward Hacking](https://mlsanity.com/paper/2510.13036/) (2025)
- [TreeC: a method to generate interpretable energy management systems using a metaheuristic algorithm](https://mlsanity.com/paper/2304.08310/) (2023)
- [Limitations of Physics-Informed Neural Networks: a Study on Smart Grid Surrogation](https://mlsanity.com/paper/2508.21559/) (2025)
- [Optimizing Energy Management of Smart Grid using Reinforcement Learning aided by Surrogate models built using Physics-informed Neural Networks](https://mlsanity.com/paper/2510.17380/) (2025)
- [MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets](https://mlsanity.com/paper/2602.16063/) (2026)
- [Topology-Aware Graph Reinforcement Learning for Energy Storage Systems Optimal Dispatch in Distribution Networks](https://mlsanity.com/paper/2603.26264/) (2026)

## Abstract (arXiv)

Active network management (ANM) of electricity distribution networks include many complex stochastic sequential optimization problems. These problems need to be solved for integrating renewable energies and distributed storage into future electrical grids. In this work, we introduce Gym-ANM, a framework for designing reinforcement learning (RL) environments that model ANM tasks in electricity distribution networks. These environments provide new playgrounds for RL research in the management of electricity networks that do not require an extensive knowledge of the underlying dynamics of such systems. Along with this work, we are releasing an implementation of an introductory toy-environment, ANM6-Easy, designed to emphasize common challenges in ANM. We also show that state-of-the-art RL algorithms can already achieve good performance on ANM6-Easy when compared against a model predictive control (MPC) approach. Finally, we provide guidelines to create new Gym-ANM environments differing in terms of (a) the distribution network topology and parameters, (b) the observation space, (c) the modelling of the stochastic processes present in the system, and (d) a set of hyperparameters influencing the reward signal. Gym-ANM can be downloaded at https://github.com/robinhenry/gym-anm.