# Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers

arXiv: https://arxiv.org/abs/2405.15557 · HTML: https://mlsanity.com/paper/2405.15557/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 47 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 11 of 11)

- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Message Passing Neural PDE Solvers](https://mlsanity.com/paper/2202.03376/) (2022)
- [Convolutional Neural Operators for robust and accurate learning of PDEs](https://mlsanity.com/paper/2302.01178/) (2023)
- [Equinox: neural networks in JAX via callable PyTrees and filtered transformations](https://mlsanity.com/paper/2111.00254/) (2021)
- [Multigrid-augmented deep learning preconditioners for the Helmholtz equation](https://mlsanity.com/paper/2203.11025/) (2022)
- [Deep Learning of Preconditioners for Conjugate Gradient Solvers in Urban Water Related Problems](https://mlsanity.com/paper/1906.06925/) (2019)
- [Neural incomplete factorization: learning preconditioners for the conjugate gradient method](https://mlsanity.com/paper/2305.16368/) (2023)
- [Neural-network preconditioners for solving the Dirac equation in lattice gauge theory](https://mlsanity.com/paper/2208.02728/) (2022)
- [Newton Informed Neural Operator for Computing Multiple Solutions of Nonlinear Partials Differential Equations](https://mlsanity.com/paper/2405.14096/) (2024)

## Built on by (most cited first, 15 of 15)

- [Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems](https://mlsanity.com/paper/2406.00809/) (2024)
- [Learning incomplete factorization preconditioners for GMRES](https://mlsanity.com/paper/2409.08262/) (2024)
- [Accurate and scalable deep Maxwell solvers using multilevel iterative methods](https://mlsanity.com/paper/2509.03622/) (2025)
- [SymMaP: Improving Computational Efficiency in Linear Solvers through Symbolic Preconditioning](https://mlsanity.com/paper/2510.24170/) (2025)
- [Message-Passing GNNs Fail to Approximate Sparse Triangular Factorizations](https://mlsanity.com/paper/2502.01397/) (2025)
- [Preconditioned Deformation Grids](https://mlsanity.com/paper/2509.18097/) (2025)
- [Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs](https://mlsanity.com/paper/2510.27517/) (2025)
- [Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences](https://mlsanity.com/paper/2602.22212/) (2026)
- [Spectral Analysis of the Weighted Frobenius Objective](https://mlsanity.com/paper/2509.16783/) (2025)
- [Factorization-in-Loop: Proximal Fill-in Minimization for Sparse Matrix Reordering](https://mlsanity.com/paper/2511.09093/) (2025)
- [Learning to Discover Iterative Spectral Algorithms](https://mlsanity.com/paper/2602.09530/) (2026)
- [RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Corrections](https://mlsanity.com/paper/2605.26854/) (2026)
- [When a neural surrogate cannot accelerate a solver: runtime share, closed-loop drift, and the economics of uncertainty gating in a stiff coupled simulation](https://mlsanity.com/paper/2608.23075/) (2026)
- [An overview of machine learning-enhanced iterative methods for systems of linear and nonlinear equations](https://mlsanity.com/paper/2610.07211/) (2026)
- [Neural Network Preconditioning: A Case Study for the Solution of the Parametric Helmholtz Equation](https://mlsanity.com/paper/2610.16542/) (2026)

## Abstract (arXiv)

Large linear systems are ubiquitous in modern computational science and engineering. The main recipe for solving them is the use of Krylov subspace iterative methods with well-designed preconditioners. Recently, GNNs have been shown to be a promising tool for designing preconditioners to reduce the overall computational cost of iterative methods by constructing them more efficiently than with classical linear algebra techniques. Preconditioners designed with these approaches cannot outperform those designed with classical methods in terms of the number of iterations in CG. In our work, we recall well-established preconditioners from linear algebra and use them as a starting point for training the GNN to obtain preconditioners that reduce the condition number of the system more significantly than classical preconditioners. Numerical experiments show that our approach outperforms both classical and neural network-based methods for an important class of parametric partial differential equations. We also provide a heuristic justification for the loss function used and show that preconditioners obtained by learning with this loss function reduce the condition number in a more desirable way for CG.