# Self-Supervised Learning of Iterative Solvers for Constrained Optimization

arXiv: https://arxiv.org/abs/2409.08066 · HTML: https://mlsanity.com/paper/2409.08066/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Learning to Optimize](https://mlsanity.com/paper/1606.01885/) (2016)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [OSQP: An Operator Splitting Solver for Quadratic Programs](https://mlsanity.com/paper/1711.08013/) (2017)
- [Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities](https://mlsanity.com/paper/2307.13565/) (2023)
- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [Self-Supervised Primal-Dual Learning for Constrained Optimization](https://mlsanity.com/paper/2208.09046/) (2022)
- [Efficient representation and approximation of model predictive control laws via deep learning](https://mlsanity.com/paper/1806.10644/) (2018)
- [Learning an Approximate Model Predictive Controller with Guarantees](https://mlsanity.com/paper/1806.04167/) (2018)
- [Large Scale Model Predictive Control with Neural Networks and Primal Active Sets](https://mlsanity.com/paper/1910.10835/) (2019)
- [End-to-End Learning to Warm-Start for Real-Time Quadratic Optimization](https://mlsanity.com/paper/2212.08260/) (2022)
- [Fast Approximations for Job Shop Scheduling: A Lagrangian Dual Deep Learning Method](https://mlsanity.com/paper/2110.06365/) (2021)
- [Safe and Fast Tracking on a Robot Manipulator: Robust MPC and Neural Network Control](https://mlsanity.com/paper/1912.10360/) (2019)
- [Reliably-stabilizing piecewise-affine neural network controllers](https://mlsanity.com/paper/2111.07183/) (2021)
- [Probabilistic performance validation of deep learning-based robust NMPC controllers](https://mlsanity.com/paper/1910.13906/) (2019)
- [Stability and feasibility of neural network-based controllers via output range analysis](https://mlsanity.com/paper/2004.00521/) (2020)
- [Neural Networks for Fast Optimisation in Model Predictive Control: A Review](https://mlsanity.com/paper/2309.02668/) (2023)
- [Deep Learning-Based Model Predictive Control for Resonant Power Converters](https://mlsanity.com/paper/1810.04872/) (2018)
- [FBstab: A Stabilized Semismooth Quadratic Programming Algorithm with Applications in Model Predictive Control](https://mlsanity.com/paper/1901.04046/) (2019)
- [An Improved Data Augmentation Scheme for Model Predictive Control Policy Approximation](https://mlsanity.com/paper/2303.05607/) (2023)
- [Error bounds for maxout neural network approximations of model predictive control](https://mlsanity.com/paper/2304.08779/) (2023)

## Built on by (most cited first, 4 of 4)

- [Learning to Optimize for Mixed-Integer Non-linear Programming with Feasibility Guarantees](https://mlsanity.com/paper/2410.11061/) (2024)
- [Generalizable Reasoning through Compositional Energy Minimization](https://mlsanity.com/paper/2510.20607/) (2025)
- [Learned Preconditioning for a Primal-Dual Interior-Point Method](https://mlsanity.com/paper/2609.35665/) (2026)
- [A Scalable Learned Iterative Solver for Nonlinear MPC via Temporal Decomposition](https://mlsanity.com/paper/2610.13480/) (2026)

## Abstract (arXiv)

The real-time solution of parametric optimization problems is critical for applications that demand high accuracy under tight real-time constraints, such as model predictive control. To this end, this work presents a learning-based iterative solver for constrained optimization, comprising a neural network predictor that generates initial primal-dual solution estimates, followed by a learned iterative solver that refines these estimates to reach high accuracy. We introduce a novel loss function based on Karush-Kuhn-Tucker (KKT) optimality conditions, enabling fully self-supervised training without pre-solved optimizer solutions. Theoretical guarantees ensure that the training loss function attains minima exclusively at KKT points. A convexification procedure enables application to nonconvex problems while preserving these guarantees. Experiments on two nonconvex case studies demonstrate speedups of up to one order of magnitude compared to state-of-the-art solvers such as IPOPT, while achieving orders of magnitude higher accuracy than competing learning-based approaches.