# A Deep Conjugate Direction Method for Iteratively Solving Linear Systems

arXiv: https://arxiv.org/abs/2205.10763 · HTML: https://mlsanity.com/paper/2205.10763/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 22 May 2022.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Learning to Optimize: A Primer and A Benchmark](https://mlsanity.com/paper/2103.12828/) (2021)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Learning to Optimize](https://mlsanity.com/paper/1606.01885/) (2016)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [Learning to Optimize Multigrid PDE Solvers](https://mlsanity.com/paper/1902.10248/) (2019)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [Learning Algebraic Multigrid Using Graph Neural Networks](https://mlsanity.com/paper/2003.05744/) (2020)
- [Deep Learning of Preconditioners for Conjugate Gradient Solvers in Urban Water Related Problems](https://mlsanity.com/paper/1906.06925/) (2019)
- [Machine-Learned Preconditioners for Linear Solvers in Geophysical Fluid Flows](https://mlsanity.com/paper/2010.02866/) (2020)
- [Accelerating GMRES with Deep Learning in Real-Time](https://mlsanity.com/paper/2103.10975/) (2021)
- [A Momentum-Conserving Implicit Material Point Method for Surface Energies with Spatial Gradients](https://mlsanity.com/paper/2101.12408/) (2021)

## Built on by (most cited first, 12 of 12)

- [Accelerating PDE Data Generation via Differential Operator Action in Solution Space](https://mlsanity.com/paper/2402.05957/) (2024)
- [Variational operator learning: A unified paradigm marrying training neural operators and solving partial differential equations](https://mlsanity.com/paper/2304.04234/) (2023)
- [Metamizer: a versatile neural optimizer for fast and accurate physics simulations](https://mlsanity.com/paper/2410.19746/) (2024)
- [Principled Acceleration of Iterative Numerical Methods Using Machine Learning](https://mlsanity.com/paper/2206.08594/) (2022)
- [Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method](https://mlsanity.com/paper/2508.00101/) (2025)
- [A Neural-Operator Preconditioned Newton Method for Accelerated Nonlinear Solvers](https://mlsanity.com/paper/2511.08811/) (2025)
- [A Neural-preconditioned Poisson Solver for Mixed Dirichlet and Neumann Boundary Conditions](https://mlsanity.com/paper/2310.00177/) (2023)
- [Neural-HSS: Hierarchical Semi-Separable Neural PDE Solver](https://mlsanity.com/paper/2602.18248/) (2026)
- [Improving Pseudo-Time Stepping Convergence for CFD Simulations With Neural Networks](https://mlsanity.com/paper/2310.06717/) (2023)
- [Learning to Discover Iterative Spectral Algorithms](https://mlsanity.com/paper/2602.09530/) (2026)
- [RL4RLA: Teaching ML to Discover Randomized Linear Algebra Algorithms Through Curriculum Design and Graph-Based Search](https://mlsanity.com/paper/2605.18004/) (2026)
- [Repairability of Inexact Solvers in Recursive State Estimation with Machine Learning](https://mlsanity.com/paper/2609.28425/) (2026)

## Abstract (arXiv)

We present a novel deep learning approach to approximate the solution of large, sparse, symmetric, positive-definite linear systems of equations. These systems arise from many problems in applied science, e.g., in numerical methods for partial differential equations. Algorithms for approximating the solution to these systems are often the bottleneck in problems that require their solution, particularly for modern applications that require many millions of unknowns. Indeed, numerical linear algebra techniques have been investigated for many decades to alleviate this computational burden. Recently, data-driven techniques have also shown promise for these problems. Motivated by the conjugate gradients algorithm that iteratively selects search directions for minimizing the matrix norm of the approximation error, we design an approach that utilizes a deep neural network to accelerate convergence via data-driven improvement of the search directions. Our method leverages a carefully chosen convolutional network to approximate the action of the inverse of the linear operator up to an arbitrary constant. We train the network using unsupervised learning with a loss function equal to the L2 difference between an input and the system matrix times the network evaluation, where the unspecified constant in the approximate inverse is accounted for. We demonstrate the efficacy of our approach on spatially discretized Poisson equations with millions of degrees of freedom arising in computational fluid dynamics applications. Unlike state-of-the-art learning approaches, our algorithm is capable of reducing the linear system residual to a given tolerance in a small number of iterations, independent of the problem size. Moreover, our method generalizes effectively to various systems beyond those encountered during training.