# Reinforcement Learning to Initialize Newton-Raphson for AC Power Flow with Quantum Annealing-Based Environment Updates

arXiv: https://arxiv.org/abs/2511.20237 · HTML: https://mlsanity.com/paper/2511.20237/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [pandapower - an Open Source Python Tool for Convenient Modeling, Analysis and Optimization of Electric Power Systems](https://mlsanity.com/paper/1709.06743/) (2017)
- [Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning](https://mlsanity.com/paper/1904.09374/) (2019)
- [Hot-Starting the Ac Power Flow with Convolutional Neural Networks](https://mlsanity.com/paper/2004.09342/) (2020)
- [Convex Relaxations of Optimal Power Flow Problems: An Illustrative Example](https://mlsanity.com/paper/1510.04330/) (2015)
- [Enhancing ACPF Analysis: Integrating Newton-Raphson Method with Gradient Descent and Computational Graphs](https://mlsanity.com/paper/2406.10390/) (2024)
- [ExaGridPF: A Parallel Power Flow Solver for Transmission and Unbalanced Distribution Systems](https://mlsanity.com/paper/1711.10579/) (2017)
- [Multistability and anomalies in oscillator models of lossy power grids](https://mlsanity.com/paper/2202.02439/) (2022)
- [A Framework for Solving Continuous Energy and Power System Problems using Adiabatic Quantum Computing](https://mlsanity.com/paper/2504.06458/) (2025)

## Built on by (most cited first, 1 of 1)

- [Newton's Lantern: A Reinforcement Learning Framework for Finetuning AC Power Flow Warm Start Models](https://mlsanity.com/paper/2605.11102/) (2026)

## Abstract (arXiv)

The Newton-Raphson (NR) method is widely used for solving power flow (PF) equations due to its quadratic convergence. However, its performance deteriorates under poor initialization or extreme operating scenarios, e.g., high levels of renewable energy penetration. We propose the use of reinforcement learning (RL) to optimize the initialization of NR, and introduce a quantum-enhanced RL environment update mechanism that addresses the combinatorially large action space at each RL timestep by formulating the voltage adjustment task as a Quadratic Unconstrained Binary Optimization (QUBO) problem, solved with an Ising machine. RL initialization is benchmarked against flat start and start from the DC (linearized) PF solution on a standard 4-bus system, Iwamoto's ill-conditioned 11-bus system, and the IEEE 118-bus system under normal and stressed loading and reactive power limits, with verified operational solutions. On all systems, a supervised initializer refined by RL requires fewer NR iterations than flat and DC starts and than the same initializer without RL, for all seeds. For example, on the 118-bus system under normal and stressed loading, it reached 2.04 and 2.86 NR iterations, compared with 3.02 and 5.13 from DC start and 2.61 and 3.09 without RL. In wall-clock time, this pays off only for an initializer integrated into the solver and reused for many solves on a fixed topology. On the 4-bus system, a quantum-enhanced RL agent with a quantum-inspired annealer moved challenging initial states that required 29 and 44 NR iterations to initializations that required three NR iterations within one RL timestep.