# Neural incomplete factorization: learning preconditioners for the conjugate gradient method

arXiv: https://arxiv.org/abs/2305.16368 · HTML: https://mlsanity.com/paper/2305.16368/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 75 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [OSQP: An Operator Splitting Solver for Quadratic Programs](https://mlsanity.com/paper/1711.08013/) (2017)
- [GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training](https://mlsanity.com/paper/2009.03294/) (2020)
- [Everything is Connected: Graph Neural Networks](https://mlsanity.com/paper/2301.08210/) (2023)
- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [Graph Neural Networks are Dynamic Programmers](https://mlsanity.com/paper/2203.15544/) (2022)
- [On Representing Linear Programs by Graph Neural Networks](https://mlsanity.com/paper/2209.12288/) (2022)
- [Multigrid-augmented deep learning preconditioners for the Helmholtz equation](https://mlsanity.com/paper/2203.11025/) (2022)
- [On Representing Mixed-Integer Linear Programs by Graph Neural Networks](https://mlsanity.com/paper/2210.10759/) (2022)
- [Deep Learning of Preconditioners for Conjugate Gradient Solvers in Urban Water Related Problems](https://mlsanity.com/paper/1906.06925/) (2019)
- [Machine-Learned Preconditioners for Linear Solvers in Geophysical Fluid Flows](https://mlsanity.com/paper/2010.02866/) (2020)
- [Accelerated Forward-Backward Optimization using Deep Learning](https://mlsanity.com/paper/2105.05210/) (2021)
- [Graph-based Neural Acceleration for Nonnegative Matrix Factorization](https://mlsanity.com/paper/2202.00264/) (2022)
- [Riemannian Metric Learning for Symmetric Positive Definite Matrices](https://mlsanity.com/paper/1501.02393/) (2015)
- [Optimized Sparse Matrix Operations for Reverse Mode Automatic Differentiation](https://mlsanity.com/paper/2212.05159/) (2022)
- [Towards Neural Sparse Linear Solvers](https://mlsanity.com/paper/2203.06944/) (2022)

## Built on by (most cited first, 18 of 18)

- [Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers](https://mlsanity.com/paper/2405.15557/) (2024)
- [Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems](https://mlsanity.com/paper/2406.00809/) (2024)
- [Neural Green's Operators for Parametric Partial Differential Equations](https://mlsanity.com/paper/2406.01857/) (2024)
- [SymMaP: Improving Computational Efficiency in Linear Solvers through Symbolic Preconditioning](https://mlsanity.com/paper/2510.24170/) (2025)
- [Learning to accelerate distributed ADMM using graph neural networks](https://mlsanity.com/paper/2509.05288/) (2025)
- [Generative modeling of Sparse Approximate Inverse Preconditioners](https://mlsanity.com/paper/2405.11007/) (2024)
- [Message-Passing GNNs Fail to Approximate Sparse Triangular Factorizations](https://mlsanity.com/paper/2502.01397/) (2025)
- [Warm-starting active-set solvers using graph neural networks](https://mlsanity.com/paper/2511.13174/) (2025)
- [Preconditioned Deformation Grids](https://mlsanity.com/paper/2509.18097/) (2025)
- [Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs](https://mlsanity.com/paper/2510.27517/) (2025)
- [Precision autotuning for linear solvers via contextual bandit-based RL](https://mlsanity.com/paper/2601.00728/) (2026)
- [Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences](https://mlsanity.com/paper/2602.22212/) (2026)
- [Learning to Solve Related Linear Systems](https://mlsanity.com/paper/2503.17265/) (2025)
- [Matrix-free Neural Preconditioner for the Dirac Operator in Lattice Gauge Theory](https://mlsanity.com/paper/2509.10378/) (2025)
- [Learning to Discover Iterative Spectral Algorithms](https://mlsanity.com/paper/2602.09530/) (2026)
- [Estimating Condition Number with Graph Neural Networks](https://mlsanity.com/paper/2603.10277/) (2026)
- [RL4RLA: Teaching ML to Discover Randomized Linear Algebra Algorithms Through Curriculum Design and Graph-Based Search](https://mlsanity.com/paper/2605.18004/) (2026)
- [RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Corrections](https://mlsanity.com/paper/2605.26854/) (2026)

## Abstract (arXiv)

The convergence of the conjugate gradient method for solving large-scale and sparse linear equation systems depends on the spectral properties of the system matrix, which can be improved by preconditioning. In this paper, we develop a computationally efficient data-driven approach to accelerate the generation of effective preconditioners. We, therefore, replace the typically hand-engineered preconditioners by the output of graph neural networks. Our method generates an incomplete factorization of the matrix and is, therefore, referred to as neural incomplete factorization (NeuralIF). Optimizing the condition number of the linear system directly is computationally infeasible. Instead, we utilize a stochastic approximation of the Frobenius loss which only requires matrix-vector multiplications for efficient training. At the core of our method is a novel message-passing block, inspired by sparse matrix theory, that aligns with the objective of finding a sparse factorization of the matrix. We evaluate our proposed method on both synthetic problem instances and on problems arising from the discretization of the Poisson equation on varying domains. Our experiments show that by using data-driven preconditioners within the conjugate gradient method we are able to speed up the convergence of the iterative procedure. The code is available at https://github.com/paulhausner/neural-incomplete-factorization.