# Accelerating Quadratic Optimization with Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.10847 · HTML: https://mlsanity.com/paper/2107.10847/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 22 Jul 2021.

## Builds on (most cited first, 24 of 24)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [Learning Combinatorial Optimization Algorithms over Graphs](https://mlsanity.com/paper/1704.01665/) (2017)
- [Learning to Optimize](https://mlsanity.com/paper/1606.01885/) (2016)
- [Learning to Perform Local Rewriting for Combinatorial Optimization](https://mlsanity.com/paper/1810.00337/) (2018)
- [OSQP: An Operator Splitting Solver for Quadratic Programs](https://mlsanity.com/paper/1711.08013/) (2017)
- [One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control](https://mlsanity.com/paper/2007.04976/) (2020)
- [Understanding and correcting pathologies in the training of learned optimizers](https://mlsanity.com/paper/1810.10180/) (2018)
- [Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization](https://mlsanity.com/paper/1910.02653/) (2019)
- [Learning to Branch](https://mlsanity.com/paper/1803.10150/) (2018)
- [Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging Problems](https://mlsanity.com/paper/2002.09611/) (2020)
- [The Voice of Optimization](https://mlsanity.com/paper/1812.09991/) (2018)
- [Linear Convergence and Metric Selection for Douglas-Rachford Splitting and ADMM](https://mlsanity.com/paper/1410.8479/) (2014)
- [GOMP: Grasp-Optimized Motion Planning for Bin Picking](https://mlsanity.com/paper/2003.02401/) (2020)

## Built on by (most cited first, 30 of 30)

- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [Automated Dynamic Algorithm Configuration](https://mlsanity.com/paper/2205.13881/) (2022)
- [Learning to Warm-Start Fixed-Point Optimization Algorithms](https://mlsanity.com/paper/2309.07835/) (2023)
- [A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases](https://mlsanity.com/paper/2209.11208/) (2022)
- [Expressive Power of Graph Neural Networks for (Mixed-Integer) Quadratic Programs](https://mlsanity.com/paper/2406.05938/) (2024)
- [Artificial Intelligence for Operations Research: Revolutionizing the Operations Research Process](https://mlsanity.com/paper/2401.03244/) (2024)
- [Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPs](https://mlsanity.com/paper/2503.12932/) (2025)
- [Learning context-aware adaptive solvers to accelerate quadratic programming](https://mlsanity.com/paper/2211.12443/) (2022)
- [Optimization for Amortized Inverse Problems](https://mlsanity.com/paper/2210.13983/) (2022)
- [A Reinforcement Learning Approach to Parameter Selection for Distributed Optimal Power Flow](https://mlsanity.com/paper/2110.11991/) (2021)
- [Data-Driven Performance Guarantees for Classical and Learned Optimizers](https://mlsanity.com/paper/2404.13831/) (2024)
- [NeuralQP: A General Hypergraph-based Optimization Framework for Large-scale QCQPs](https://mlsanity.com/paper/2410.03720/) (2024)
- [Towards graph neural networks for provably solving convex optimization problems](https://mlsanity.com/paper/2502.02446/) (2025)
- [Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding](https://mlsanity.com/paper/2512.01565/) (2025)
- [Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms](https://mlsanity.com/paper/2508.00775/) (2025)
- [Learning to accelerate distributed ADMM using graph neural networks](https://mlsanity.com/paper/2509.05288/) (2025)
- [Hint assisted reinforcement learning: an application in radio astronomy](https://mlsanity.com/paper/2301.03933/) (2023)
- [Warm-starting active-set solvers using graph neural networks](https://mlsanity.com/paper/2511.13174/) (2025)
- [Learning to accelerate Krasnosel'skii-Mann fixed-point iterations with guarantees](https://mlsanity.com/paper/2601.07665/) (2026)
- [Scalable Deep Unfolding of Conic Optimizers](https://mlsanity.com/paper/2606.13825/) (2026)
- [NICE: Robust Scheduling through Reinforcement Learning-Guided Integer Programming](https://mlsanity.com/paper/2109.12171/) (2021)
- [Bridging the gap between QP-based and MPC-based RL](https://mlsanity.com/paper/2205.08856/) (2022)
- [Parametric Nonconvex Optimization via Convex Surrogates](https://mlsanity.com/paper/2604.05640/) (2026)
- [Multi-Abstractive Neural Controller: An Efficient Hierarchical Control Architecture for Interactive Driving](https://mlsanity.com/paper/2305.14797/) (2023)
- [Reinforcement learning for adaptive interior point methods in convex quadratic programming](https://mlsanity.com/paper/2509.07404/) (2025)
- [Data-driven Projection Generation for Efficiently Solving Heterogeneous Quadratic Programming Problems](https://mlsanity.com/paper/2510.26061/) (2025)
- [Learning Over-Relaxation Policies for ADMM with Convergence Guarantees](https://mlsanity.com/paper/2604.26932/) (2026)
- [When Do Learned Diffusion Proposals Help Constraint Solving? A Controlled Study on Continuous Algebraic Systems](https://mlsanity.com/paper/2607.27169/) (2026)
- [Reinforcement Learning in Operational Research: A Technical Review and Practical Roadmap](https://mlsanity.com/paper/2609.24750/) (2026)
- [Reinforcement Learning to Accelerate Primal-Dual Hybrid Gradient for Linear Programming](https://mlsanity.com/paper/2610.01546/) (2026)

## Abstract (arXiv)

First-order methods for quadratic optimization such as OSQP are widely used for large-scale machine learning and embedded optimal control, where many related problems must be rapidly solved. These methods face two persistent challenges: manual hyperparameter tuning and convergence time to high-accuracy solutions. To address these, we explore how Reinforcement Learning (RL) can learn a policy to tune parameters to accelerate convergence. In experiments with well-known QP benchmarks we find that our RL policy, RLQP, significantly outperforms state-of-the-art QP solvers by up to 3x. RLQP generalizes surprisingly well to previously unseen problems with varying dimension and structure from different applications, including the QPLIB, Netlib LP and Maros-Meszaros problems. Code for RLQP is available at https://github.com/berkeleyautomation/rlqp.