# A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow

arXiv: https://arxiv.org/abs/2110.11991 · HTML: https://mlsanity.com/paper/2110.11991/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 22 Oct 2021.

## Builds on (most cited first, 17 of 17)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Finite-Sample Analysis of Nonlinear Stochastic Approximation with Applications in Reinforcement Learning](https://mlsanity.com/paper/1905.11425/) (2019)
- [Dealing with Sparse Rewards in Reinforcement Learning](https://mlsanity.com/paper/1910.09281/) (2019)
- [Accelerating Quadratic Optimization with Reinforcement Learning](https://mlsanity.com/paper/2107.10847/) (2021)
- [Differentiable Linearized ADMM](https://mlsanity.com/paper/1905.06179/) (2019)
- [Strong NP-hardness of AC power flows feasibility](https://mlsanity.com/paper/1512.07315/) (2015)
- [Adaptive ADMM with Spectral Penalty Parameter Selection](https://mlsanity.com/paper/1605.07246/) (2016)
- [A Two-level ADMM Algorithm for AC OPF with Global Convergence Guarantees](https://mlsanity.com/paper/2008.12139/) (2020)
- [A Privacy-Preserving Distributed Control of Optimal Power Flow](https://mlsanity.com/paper/2102.02276/) (2021)
- [Differentially Private Distributed Optimal Power Flow](https://mlsanity.com/paper/1910.10136/) (2019)
- [A Component-Based Dual Decomposition Method for the OPF Problem](https://mlsanity.com/paper/1704.03647/) (2017)
- [Towards Distributed OPF using ALADIN](https://mlsanity.com/paper/1802.08603/) (2018)
- [Leveraging GPU batching for scalable nonlinear programming through massive Lagrangian decomposition](https://mlsanity.com/paper/2106.14995/) (2021)

## Built on by (most cited first, 5 of 5)

- [Artificial Intelligence for Operations Research: Revolutionizing the Operations Research Process](https://mlsanity.com/paper/2401.03244/) (2024)
- [ORGEval: Graph-Theoretic Evaluation of LLMs in Optimization Modeling](https://mlsanity.com/paper/2510.27610/) (2025)
- [Taking the human out of decomposition-based optimization via artificial intelligence: Part II. Learning to initialize](https://mlsanity.com/paper/2310.07082/) (2023)
- [QCQP-Net: Reliably Learning Feasible Alternating Current Optimal Power Flow Solutions Under Constraints](https://mlsanity.com/paper/2401.06820/) (2024)
- [Accelerating Multi-Block Constrained Optimization Through Learning to Optimize](https://mlsanity.com/paper/2409.17320/) (2024)

## Abstract (arXiv)

With the increasing penetration of distributed energy resources, distributed optimization algorithms have attracted significant attention for power systems applications due to their potential for superior scalability, privacy, and robustness to a single point-of-failure. The Alternating Direction Method of Multipliers (ADMM) is a popular distributed optimization algorithm; however, its convergence performance is highly dependent on the selection of penalty parameters, which are usually chosen heuristically. In this work, we use reinforcement learning (RL) to develop an adaptive penalty parameter selection policy for the AC optimal power flow (ACOPF) problem solved via ADMM with the goal of minimizing the number of iterations until convergence. We train our RL policy using deep Q-learning, and show that this policy can result in significantly accelerated convergence (up to a 59% reduction in the number of iterations compared to existing, curvature-informed penalty parameter selection methods). Furthermore, we show that our RL policy demonstrates promise for generalizability, performing well under unseen loading schemes as well as under unseen losses of lines and generators (up to a 50% reduction in iterations). This work thus provides a proof-of-concept for using RL for parameter selection in ADMM for power systems applications.