# Optimal Tap Setting of Voltage Regulation Transformers Using Batch Reinforcement Learning

arXiv: https://arxiv.org/abs/1807.10997 · HTML: https://mlsanity.com/paper/1807.10997/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2018.

## Built on by (most cited first, 8 of 8)

- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems](https://mlsanity.com/paper/2111.05969/) (2021)
- [A Multi-scale Time-series Dataset with Benchmark for Machine Learning in Decarbonized Energy Grids](https://mlsanity.com/paper/2110.06324/) (2021)
- [Multi-Stage Transmission Line Flow Control Using Centralized and Decentralized Reinforcement Learning Agents](https://mlsanity.com/paper/2102.08430/) (2021)
- [Robust, Deep, and Reinforcement Learning for Management of Communication and Power Networks](https://mlsanity.com/paper/2202.05395/) (2022)
- [A Reinforcement Learning-based Volt-VAR Control Dataset and Testing Environment](https://mlsanity.com/paper/2204.09500/) (2022)
- [Energy System Digitization in the Era of AI: A Three-Layered Approach towards Carbon Neutrality](https://mlsanity.com/paper/2211.04584/) (2022)
- [Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control](https://mlsanity.com/paper/2410.15188/) (2024)

## Abstract (arXiv)

In this paper, we address the problem of setting the tap positions of load tap changers (LTCs) for voltage regulation in radial power distribution systems under uncertain load dynamics. The objective is to find a policy to determine the tap positions that only uses measurements of voltage magnitudes and topology information so as to minimize the voltage deviation across the system. We formulate this problem as a Markov decision process (MDP), and propose a batch reinforcement learning (RL) algorithm to solve it. By taking advantage of a linearized power flow model, we propose an effective algorithm to estimate the voltage magnitudes under different tap settings, which allows the RL algorithm to explore the state and action spaces freely offline without impacting the system operation. To circumvent the "curse of dimensionality" resulted from the large state and action spaces, we propose a sequential learning algorithm to learn an action-value function for each LTC, based on which the optimal tap positions can be directly determined. The effectiveness of the proposed algorithm is validated via numerical simulations on the IEEE 13-bus and 123-bus distribution test feeders.