mlsanityData to 14 Oct 2026

Learning to Warm-Start Fixed-Point Optimization Algorithms

arXiv 2309.07835 ↗First posted 14 Sep 2023math.OC cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

We introduce a machine-learning framework to warm-start fixed-point optimization algorithms. Our architecture consists of a neural network mapping problem parameters to warm starts, followed by a predefined number of fixed-point iterations. We propose two loss functions designed to either minimize the fixed-point residual or the distance to a ground truth solution. In this way, the neural network predicts warm starts with the end-to-end goal of minimizing the downstream loss. An important feature of our architecture is its flexibility, in that it can predict a warm start for fixed-point algorithms run for any number of steps, without being limited to the number of steps it has been trained on. We provide PAC-Bayes generalization bounds on unseen data for common classes of fixed-point operators: contractive, linearly convergent, and averaged. Applying this framework to well-known applications in control, statistics, and signal processing, we observe a significant reduction in the number of iterations and solution time required to solve these problems, through learned warm starts.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 63 of its 123 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

63 of 123 references.

References · 63

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  4. 2015Automatic differentiation in machine learning: a surveycited by 855
  5. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  6. 2016Learning to learn by gradient descent by gradient descentcited by 620
  7. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 421
  8. 2017Learning Deep CNN Denoiser Prior for Image Restorationcited by 408
  9. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  10. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  11. 2017EMNIST: an extension of MNIST to handwritten letterscited by 213
  12. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
38 more
  1. 2016Learning to Optimizecited by 195
  2. 2017The Case for Learned Index Structurescited by 188
  3. 2017OSQP: An Operator Splitting Solver for Quadratic Programscited by 161
  4. 2020Algorithms with Predictionscited by 153
  5. 2015Global Convergence of ADMM in Nonconvex Nonsmooth Optimizationcited by 98
  6. 2022Tutorial on amortized optimizationcited by 83
  7. 2019Learning Optimal Solutions for Extremely Fast AC Optimal Power Flowcited by 77
  8. 2019Near-Optimal Bounds for Online Caching with Machine Learned Advicecited by 73
  9. 2017One Network to Solve Them All --- Solving Linear Inverse Problems using Deep Projection Modelscited by 67
  10. 2014Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problemscited by 64
  11. 2015A PAC Approach to Application-Specific Algorithm Selectioncited by 58
  12. 2018Efficient representation and approximation of model predictive control laws via deep learningcited by 58
  13. 2022VeLO: Training Versatile Learned Optimizers by Scaling Upcited by 57
  14. 2021Faster Matchings via Learned Dualscited by 56
  15. 2019Deep Unfolding for Communications Systems: A Survey and Some New Directionscited by 49
  16. 2021Acceleration Methodscited by 48
  17. 2017Unrolled Optimization with Deep Priorscited by 40
  18. 2019Online Mixed-Integer Optimization in Millisecondscited by 38
  19. 2015Smooth Strongly Convex Interpolation and Exact Worst-case Performance of First-order Methodscited by 37
  20. 2018Learning for Constrained Optimization: Identifying Optimal Active Constraint Setscited by 36
  21. 2019Scalable Semidefinite Programmingcited by 36
  22. 2019Learning Warm-Start Points for AC Optimal Power Flowcited by 32
  23. 2016Learning-Theoretic Foundations of Algorithm Configuration for Combinatorial Partitioning Problemscited by 30
  24. 2021Accelerating Quadratic Optimization with Reinforcement Learningcited by 30
  25. 2019Large Scale Model Predictive Control with Neural Networks and Primal Active Setscited by 29
  26. 2018The Voice of Optimizationcited by 28
  27. 2013Phase Retrieval from Coded Diffraction Patternscited by 26
  28. 2022End-to-End Learning to Warm-Start for Real-Time Quadratic Optimizationcited by 26
  29. 2018Globally Convergent Type-I Anderson Acceleration for Non-Smooth Fixed-Point Iterationscited by 25
  30. 2022Learning Predictions for Algorithms with Predictionscited by 25
  31. 2020Safeguarded Learned Convex Optimizationcited by 24
  32. 2022Generalization Bounds for Data-Driven Numerical Linear Algebracited by 24
  33. 2022Learning to Solve Optimization Problems with Hard Linear Constraintscited by 23
  34. 2020Automatically Learning Compact Quality-aware Surrogates for Optimization Problemscited by 22
  35. 2017Multi-Period Trading via Convex Optimizationcited by 21
  36. 2018Solution Refinement at Regular Points of Conic Problemscited by 21
  37. 2020Operator splitting for a homogeneous embedding of the linear complementarity problemcited by 21
  38. 2021Policy Search for Model Predictive Control with Application to Agile Drone Flightcited by 20

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 11; 2026: 13.

32 papers.

Sort:

By area:

  1. 2025FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guaranteescited by 21
  2. 2025Pinet: Optimizing hard-constrained neural networks with orthogonal projection layerscited by 19
  3. 2024Differentiation Through Black-Box Quadratic Programming Solverscited by 10
  4. 2025Aligning Diffusion Model with Problem Constraints for Trajectory Optimizationcited by 7
  5. 2024Data-Driven Performance Guarantees for Classical and Learned Optimizerscited by 5
  6. 2024Learning Multiple Initial Solutions to Optimization Problemscited by 4
  7. 2025A First-order Generative Bilevel Optimization Framework for Diffusion Modelscited by 4
  8. 2025Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfoldingcited by 4
  9. 2025Learning to optimize with guarantees: a complete characterization of linearly convergent algorithmscited by 3
  10. 2025Learning to accelerate distributed ADMM using graph neural networkscited by 3
  11. 2024From Learning to Optimize to Learning Optimization Algorithmscited by 2
  12. 2024WARP-LCA: Efficient Convolutional Sparse Coding with Locally Competitive Algorithmcited by 2
20 more
  1. 2026Learning to accelerate Krasnosel'skii-Mann fixed-point iterations with guaranteescited by 2
  2. 2026Scalable Deep Unfolding of Conic Optimizerscited by 2
  3. 2024Robotic warehousing operations: a learn-then-optimize approach to large-scale neighborhood searchcited by 1
  4. 2026Cheap Thrills: Effective Amortized Optimization Using Inexpensive Labelscited by 1
  5. 2026Parametric Nonconvex Optimization via Convex Surrogatescited by 1
  6. 2026GLENS: Global Search via Learning from Solver Iterates with Diffusion Modelscited by 1
  7. 2026LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimizationcited by 1
  8. 2024Large Language Models for Knowledge-Free Network Management: Feasibility Study and Opportunitiescited by 0
  9. 2024Preserving Expert-Level Privacy in Offline Reinforcement Learningcited by 0
  10. 2025Learning to Solve Constrained Bilevel Control Co-Design Problemscited by 0
  11. 2025Reinforcement learning for adaptive interior point methods in convex quadratic programmingcited by 0
  12. 2025Flow-Opt: Scalable Centralized Multi-Robot Trajectory Optimization with Flow Matching and Differentiable Optimizationcited by 0
  13. 2025KANFormer for Predicting Fill Probabilities via Survival Analysis in Limit Order Bookscited by 0
  14. 2026partialCBDs: Differentiable Causal Block Diagramscited by 0
  15. 2026Learning Over-Relaxation Policies for ADMM with Convergence Guaranteescited by 0
  16. 2026WARP: A Benchmark for Primal-Dual Warm-Starting of Interior-Point Solverscited by 0
  17. 2026Distribution-Aware Algorithm Design with LLM Agentscited by 0
  18. 2026Memory-Computation Tradeoffs in Semi Amortized Parametric Optimizationcited by 0
  19. 2026FunL2O: LLM-Guided Feature Function Design for Learning to Optimizecited by 0
  20. 2026Continuous Manifold-Decomposed Impedance Retargeting for Contact-Rich Imitation Learningcited by 0