# Accelerating GMRES with Deep Learning in Real-Time

arXiv: https://arxiv.org/abs/2103.10975 · HTML: https://mlsanity.com/paper/2103.10975/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 19 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective](https://mlsanity.com/paper/1811.03402/) (2018)
- [Using Machine Learning to Augment Coarse-Grid Computational Fluid Dynamics Simulations](https://mlsanity.com/paper/2010.00072/) (2020)
- [Evolutionary reinforcement learning of dynamical large deviations](https://mlsanity.com/paper/1909.00835/) (2019)
- [Learning to grow: control of material self-assembly using evolutionary reinforcement learning](https://mlsanity.com/paper/1912.08333/) (2019)

## Built on by (most cited first, 8 of 8)

- [A Deep Conjugate Direction Method for Iteratively Solving Linear Systems](https://mlsanity.com/paper/2205.10763/) (2022)
- [Principled Acceleration of Iterative Numerical Methods Using Machine Learning](https://mlsanity.com/paper/2206.08594/) (2022)
- [Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method](https://mlsanity.com/paper/2508.00101/) (2025)
- [A Deep Learning-Based Particle-in-Cell Method for Plasma Simulations](https://mlsanity.com/paper/2107.02232/) (2021)
- [Towards Neural Sparse Linear Solvers](https://mlsanity.com/paper/2203.06944/) (2022)
- [Neural-HSS: Hierarchical Semi-Separable Neural PDE Solver](https://mlsanity.com/paper/2602.18248/) (2026)
- [A Hybrid Kernel-Free Boundary Integral Method with Operator Learning for Solving Parametric Partial Differential Equations In Complex Domains](https://mlsanity.com/paper/2404.15242/) (2024)
- [An overview of machine learning-enhanced iterative methods for systems of linear and nonlinear equations](https://mlsanity.com/paper/2610.07211/) (2026)

## Abstract (arXiv)

GMRES is a powerful numerical solver used to find solutions to extremely large systems of linear equations. These systems of equations appear in many applications in science and engineering. Here we demonstrate a real-time machine learning algorithm that can be used to accelerate the time-to-solution for GMRES. Our framework is novel in that is integrates the deep learning algorithm in an in situ fashion: the AI-accelerator gradually learns how to optimizes the time to solution without requiring user input (such as a pre-trained data set). We describe how our algorithm collects data and optimizes GMRES. We demonstrate our algorithm by implementing an accelerated (MLGMRES) solver in Python. We then use MLGMRES to accelerate a solver for the Poisson equation -- a class of linear problems that appears in may applications. Informed by the properties of formal solutions to the Poisson equation, we test the performance of different neural networks. Our key takeaway is that networks which are capable of learning non-local relationships perform well, without needing to be scaled with the input problem size, making them good candidates for the extremely large problems encountered in high-performance computing. For the inputs studied, our method provides a roughly 2× acceleration.