# Principled Acceleration of Iterative Numerical Methods Using Machine Learning

arXiv: https://arxiv.org/abs/2206.08594 · HTML: https://mlsanity.com/paper/2206.08594/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [Learning Algebraic Multigrid Using Graph Neural Networks](https://mlsanity.com/paper/2003.05744/) (2020)
- [Path Planning using Neural A* Search](https://mlsanity.com/paper/2009.07476/) (2020)
- [Meta-learning PINN loss functions](https://mlsanity.com/paper/2107.05544/) (2021)
- [SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python](https://mlsanity.com/paper/1907.10121/) (2019)
- [Meta-Auto-Decoder for Solving Parametric Partial Differential Equations](https://mlsanity.com/paper/2111.08823/) (2021)
- [A novel meta-learning initialization method for physics-informed neural networks](https://mlsanity.com/paper/2107.10991/) (2021)
- [Multigrid-augmented deep learning preconditioners for the Helmholtz equation](https://mlsanity.com/paper/2203.11025/) (2022)
- [Poisson CNN: Convolutional neural networks for the solution of the Poisson equation on a Cartesian mesh](https://mlsanity.com/paper/1910.08613/) (2019)
- [Optimization-Based Algebraic Multigrid Coarsening Using Reinforcement Learning](https://mlsanity.com/paper/2106.01854/) (2021)
- [Meta-learning Pseudo-differential Operators with Deep Neural Networks](https://mlsanity.com/paper/1906.06782/) (2019)
- [A Helmholtz equation solver using unsupervised learning: Application to transcranial ultrasound](https://mlsanity.com/paper/2010.15761/) (2020)
- [A Deep Conjugate Direction Method for Iteratively Solving Linear Systems](https://mlsanity.com/paper/2205.10763/) (2022)
- [Neural Fixed-Point Acceleration for Convex Optimization](https://mlsanity.com/paper/2107.10254/) (2021)
- [Using neural networks to solve the 2D Poisson equation for electric field computation in plasma fluid simulations](https://mlsanity.com/paper/2109.13076/) (2021)
- [Personalized Algorithm Generation: A Case Study in Learning ODE Integrators](https://mlsanity.com/paper/2105.01303/) (2021)
- [Accelerating GMRES with Deep Learning in Real-Time](https://mlsanity.com/paper/2103.10975/) (2021)
- [Structural engineering from an inverse problems perspective](https://mlsanity.com/paper/2106.15177/) (2021)

## Built on by (most cited first, 6 of 6)

- [Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method](https://mlsanity.com/paper/2508.00101/) (2025)
- [PRISM: Distribution-free Adaptive Computation of Matrix Functions for Accelerating Neural Network Training](https://mlsanity.com/paper/2601.22137/) (2026)
- [Learning to accelerate distributed ADMM using graph neural networks](https://mlsanity.com/paper/2509.05288/) (2025)
- [Mitigating distribution shift in machine learning-augmented hybrid simulation](https://mlsanity.com/paper/2401.09259/) (2024)
- [Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving](https://mlsanity.com/paper/2405.02952/) (2024)
- [Learning to Relax: Setting Solver Parameters Across a Sequence of Linear System Instances](https://mlsanity.com/paper/2310.02246/) (2023)

## Abstract (arXiv)

Iterative methods are ubiquitous in large-scale scientific computing applications, and a number of approaches based on meta-learning have been recently proposed to accelerate them. However, a systematic study of these approaches and how they differ from meta-learning is lacking. In this paper, we propose a framework to analyze such learning-based acceleration approaches, where one can immediately identify a departure from classical meta-learning. We show that this departure may lead to arbitrary deterioration of model performance. Based on our analysis, we introduce a novel training method for learning-based acceleration of iterative methods. Furthermore, we theoretically prove that the proposed method improves upon the existing methods, and demonstrate its significant advantage and versatility through various numerical applications.