# Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach with Safe Gradient Flow

arXiv: https://arxiv.org/abs/2303.11417 · HTML: https://mlsanity.com/paper/2303.11417/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2023.

## Built on by (most cited first, 5 of 5)

- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Combining Neural Networks and Symbolic Regression for Analytical Lyapunov Function Discovery](https://mlsanity.com/paper/2406.15675/) (2024)
- [DiffOP: Reinforcement Learning of Optimization-Based Control Policies via Implicit Policy Gradients](https://mlsanity.com/paper/2411.07484/) (2024)
- [Analytical Lyapunov Function Discovery: An RL-based Generative Approach](https://mlsanity.com/paper/2502.02014/) (2025)
- [Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamics](https://mlsanity.com/paper/2605.30461/) (2026)

## Abstract (arXiv)

Deep reinforcement learning approaches are becoming appealing for the design of nonlinear controllers for voltage control problems, but the lack of stability guarantees hinders their deployment in real-world scenarios. This paper constructs a decentralized RL-based controller featuring two components: a transient control policy and a steady-state performance optimizer. The transient policy is parameterized as a neural network, and the steady-state optimizer represents the gradient of the long-term operating cost function. The two parts are synthesized through a safe gradient flow framework, which prevents the violation of reactive power capacity constraints. We prove that if the output of the transient controller is bounded and monotonically decreasing with respect to its input, then the closed-loop system is asymptotically stable and converges to the optimal steady-state solution. We demonstrate the effectiveness of our method by conducting experiments with IEEE 13-bus and 123-bus distribution system test feeders.