# Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems

arXiv: https://arxiv.org/abs/2406.00809 · HTML: https://mlsanity.com/paper/2406.00809/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 52 references that are in the map.
- First posted on arXiv 2 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling](https://mlsanity.com/paper/1801.10247/) (2018)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training](https://mlsanity.com/paper/2305.14342/) (2023)
- [Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining](https://mlsanity.com/paper/2110.08450/) (2021)
- [Stochastic Gradient Descent with Biased but Consistent Gradient Estimators](https://mlsanity.com/paper/1807.11880/) (2018)
- [Neural incomplete factorization: learning preconditioners for the conjugate gradient method](https://mlsanity.com/paper/2305.16368/) (2023)
- [Communication-Efficient Graph Neural Networks with Probabilistic Neighborhood Expansion Analysis and Caching](https://mlsanity.com/paper/2305.03152/) (2023)
- [Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers](https://mlsanity.com/paper/2405.15557/) (2024)
- [Linear-Cost Covariance Functions for Gaussian Random Fields](https://mlsanity.com/paper/1711.05895/) (2017)
- [Ising on the Graph: Task-specific Graph Subsampling via the Ising Model](https://mlsanity.com/paper/2402.10206/) (2024)

## Built on by (most cited first, 15 of 15)

- [Learning incomplete factorization preconditioners for GMRES](https://mlsanity.com/paper/2409.08262/) (2024)
- [PEARL: Preconditioner Enhancement through Actor-critic Reinforcement Learning](https://mlsanity.com/paper/2501.10750/) (2025)
- [torch-sla: Differentiable Sparse Linear Algebra with Adjoint Solvers and Sparse Tensor Parallelism for PyTorch](https://mlsanity.com/paper/2601.13994/) (2026)
- [Message-Passing GNNs Fail to Approximate Sparse Triangular Factorizations](https://mlsanity.com/paper/2502.01397/) (2025)
- [MD-PNOP: Equation-Recast Neural Operators for Minimal-Data Extrapolation and PDE Solver Acceleration](https://mlsanity.com/paper/2509.01416/) (2025)
- [Preconditioned Deformation Grids](https://mlsanity.com/paper/2509.18097/) (2025)
- [Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs](https://mlsanity.com/paper/2510.27517/) (2025)
- [Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences](https://mlsanity.com/paper/2602.22212/) (2026)
- [Equation Recast for Canonical Operator Learning Across Parametric PDEs](https://mlsanity.com/paper/2609.02982/) (2026)
- [Accelerated Portfolio Optimization and Option Pricing with Reinforcement Learning](https://mlsanity.com/paper/2507.01972/) (2025)
- [Matrix-free Neural Preconditioner for the Dirac Operator in Lattice Gauge Theory](https://mlsanity.com/paper/2509.10378/) (2025)
- [Fast Linear Solvers via AI-Tuned Markov Chain Monte Carlo-based Matrix Inversion](https://mlsanity.com/paper/2509.18452/) (2025)
- [A level-wise training scheme for learning neural multigrid smoothers with application to integral equations](https://mlsanity.com/paper/2603.01064/) (2026)
- [RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Corrections](https://mlsanity.com/paper/2605.26854/) (2026)
- [Graph Neural Multilevel Preconditioners for Iterative Solvers](https://mlsanity.com/paper/2607.28456/) (2026)

## Abstract (arXiv)

Preconditioning is at the heart of iterative solutions of large, sparse linear systems of equations in scientific disciplines. Several algebraic approaches, which access no information beyond the matrix itself, are widely studied and used, but ill-conditioned matrices remain very challenging. We take a machine learning approach and propose using graph neural networks as a general-purpose preconditioner. They show attractive performance for many problems and can be used when the mainstream preconditioners perform poorly. Empirical evaluation on over 800 matrices suggests that the construction time of these graph neural preconditioners (GNPs) is more predictable and can be much shorter than that of other widely used ones, such as ILU and AMG, while the execution time is faster than using a Krylov method as the preconditioner, such as in inner-outer GMRES. GNPs have a strong potential for solving large-scale, challenging algebraic problems arising from not only partial differential equations, but also economics, statistics, graph, and optimization, to name a few.