# Graph Neural Networks and Applied Linear Algebra

arXiv: https://arxiv.org/abs/2310.14084 · HTML: https://mlsanity.com/paper/2310.14084/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 21 Oct 2023.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Learning Algebraic Multigrid Using Graph Neural Networks](https://mlsanity.com/paper/2003.05744/) (2020)
- [Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpoint](https://mlsanity.com/paper/1912.04862/) (2019)
- [Backpropagation-Friendly Eigendecomposition](https://mlsanity.com/paper/1906.09023/) (2019)
- [Optimization-Based Algebraic Multigrid Coarsening Using Reinforcement Learning](https://mlsanity.com/paper/2106.01854/) (2021)
- [Scalable algorithms for physics-informed neural and graph networks](https://mlsanity.com/paper/2205.08332/) (2022)
- [MG-GNN: Multigrid Graph Neural Networks for Learning Multilevel Domain Decomposition Methods](https://mlsanity.com/paper/2301.11378/) (2023)
- [Learning Interface Conditions in Domain Decomposition Solvers](https://mlsanity.com/paper/2205.09833/) (2022)

## Built on by (most cited first, 6 of 6)

- [Learning incomplete factorization preconditioners for GMRES](https://mlsanity.com/paper/2409.08262/) (2024)
- [Message-Passing GNNs Fail to Approximate Sparse Triangular Factorizations](https://mlsanity.com/paper/2502.01397/) (2025)
- [Advancing Heatwave Forecasting via Distribution Informed-Graph Neural Networks (DI-GNNs): Integrating Extreme Value Theory with GNNs](https://mlsanity.com/paper/2411.13496/) (2024)
- [Deep Learning-Enhanced Preconditioning for Efficient Conjugate Gradient Solvers in Large-Scale PDE Systems](https://mlsanity.com/paper/2412.07127/) (2024)
- [Domain-Decomposed Graph Neural Network Surrogate Modeling for Ice Sheets](https://mlsanity.com/paper/2512.01888/) (2025)
- [Automated Grammar-based Algebraic Multigrid Design With Evolutionary Algorithms](https://mlsanity.com/paper/2603.17641/) (2026)

## Abstract (arXiv)

Sparse matrix computations are ubiquitous in scientific computing. With the recent interest in scientific machine learning, it is natural to ask how sparse matrix computations can leverage neural networks (NN). Unfortunately, multi-layer perceptron (MLP) neural networks are typically not natural for either graph or sparse matrix computations. The issue lies with the fact that MLPs require fixed-sized inputs while scientific applications generally generate sparse matrices with arbitrary dimensions and a wide range of nonzero patterns (or matrix graph vertex interconnections). While convolutional NNs could possibly address matrix graphs where all vertices have the same number of nearest neighbors, a more general approach is needed for arbitrary sparse matrices, e.g. arising from discretized partial differential equations on unstructured meshes. Graph neural networks (GNNs) are one approach suitable to sparse matrices. GNNs define aggregation functions (e.g., summations) that operate on variable size input data to produce data of a fixed output size so that MLPs can be applied. The goal of this paper is to provide an introduction to GNNs for a numerical linear algebra audience. Concrete examples are provided to illustrate how many common linear algebra tasks can be accomplished using GNNs. We focus on iterative methods that employ computational kernels such as matrix-vector products, interpolation, relaxation methods, and strength-of-connection measures. Our GNN examples include cases where parameters are determined a-priori as well as cases where parameters must be learned. The intent with this article is to help computational scientists understand how GNNs can be used to adapt machine learning concepts to computational tasks associated with sparse matrices. It is hoped that this understanding will stimulate data-driven extensions of classical sparse linear algebra tasks.