# Neural Fixed-Point Acceleration for Convex Optimization

arXiv: https://arxiv.org/abs/2107.10254 · HTML: https://mlsanity.com/paper/2107.10254/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Learning Combinatorial Optimization Algorithms over Graphs](https://mlsanity.com/paper/1704.01665/) (2017)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Meta-Learning with Differentiable Convex Optimization](https://mlsanity.com/paper/1904.03758/) (2019)
- [Learning to Optimize](https://mlsanity.com/paper/1606.01885/) (2016)
- [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [Solving Mixed Integer Programs Using Neural Networks](https://mlsanity.com/paper/2012.13349/) (2020)
- [Learning Latent Permutations with Gumbel-Sinkhorn Networks](https://mlsanity.com/paper/1802.08665/) (2018)
- [Multiscale Deep Equilibrium Models](https://mlsanity.com/paper/2006.08656/) (2020)
- [On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization](https://mlsanity.com/paper/1607.05447/) (2016)
- [Learned Optimizers that Scale and Generalize](https://mlsanity.com/paper/1703.04813/) (2017)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [Hypersolvers: Toward Fast Continuous-Depth Models](https://mlsanity.com/paper/2007.09601/) (2020)
- [Globally Convergent Type-I Anderson Acceleration for Non-Smooth Fixed-Point Iterations](https://mlsanity.com/paper/1808.03971/) (2018)
- [Solution Refinement at Regular Points of Conic Problems](https://mlsanity.com/paper/1811.02157/) (2018)
- [Training Learned Optimizers with Randomly Initialized Learned Optimizers](https://mlsanity.com/paper/2101.07367/) (2021)
- [Using learned optimizers to make models robust to input noise](https://mlsanity.com/paper/1906.03367/) (2019)
- [A Learning-boosted Quasi-Newton Method for AC Optimal Power Flow](https://mlsanity.com/paper/2007.06074/) (2020)
- [A Semismooth Newton Method for Fast, Generic Convex Programming](https://mlsanity.com/paper/1705.00772/) (2017)

## Built on by (most cited first, 10 of 10)

- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [Learning to Warm-Start Fixed-Point Optimization Algorithms](https://mlsanity.com/paper/2309.07835/) (2023)
- [Machine Learning for Large-Scale Optimization in 6G Wireless Networks](https://mlsanity.com/paper/2301.03377/) (2023)
- [Graph-based Neural Acceleration for Nonnegative Matrix Factorization](https://mlsanity.com/paper/2202.00264/) (2022)
- [A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms](https://mlsanity.com/paper/2211.12386/) (2022)
- [Principled Acceleration of Iterative Numerical Methods Using Machine Learning](https://mlsanity.com/paper/2206.08594/) (2022)
- [Data-Driven Performance Guarantees for Classical and Learned Optimizers](https://mlsanity.com/paper/2404.13831/) (2024)
- [Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving](https://mlsanity.com/paper/2405.02952/) (2024)
- [Memory-Computation Tradeoffs in Semi Amortized Parametric Optimization](https://mlsanity.com/paper/2607.20769/) (2026)
- [Reinforcement Learning to Accelerate Primal-Dual Hybrid Gradient for Linear Programming](https://mlsanity.com/paper/2610.01546/) (2026)

## Abstract (arXiv)

Fixed-point iterations are at the heart of numerical computing and are often a computational bottleneck in real-time applications that typically need a fast solution of moderate accuracy. We present neural fixed-point acceleration which combines ideas from meta-learning and classical acceleration methods to automatically learn to accelerate fixed-point problems that are drawn from a distribution. We apply our framework to SCS, the state-of-the-art solver for convex cone programming, and design models and loss functions to overcome the challenges of learning over unrolled optimization and acceleration instabilities. Our work brings neural acceleration into any optimization problem expressible with CVXPY. The source code behind this paper is available at https://github.com/facebookresearch/neural-scs