# Convergent Graph Solvers

arXiv: https://arxiv.org/abs/2106.01680 · HTML: https://mlsanity.com/paper/2106.01680/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 3 Jun 2021.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification](https://mlsanity.com/paper/1602.02068/) (2016)
- [Learning Convolutional Neural Networks for Graphs](https://mlsanity.com/paper/1605.05273/) (2016)
- [Efficient and Modular Implicit Differentiation](https://mlsanity.com/paper/2105.15183/) (2021)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training](https://mlsanity.com/paper/2009.03294/) (2020)
- [On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization](https://mlsanity.com/paper/1607.05447/) (2016)
- [Learning Stable Deep Dynamics Models](https://mlsanity.com/paper/2001.06116/) (2020)
- [Monotone operator equilibrium networks](https://mlsanity.com/paper/2006.08591/) (2020)
- [Implicit Graph Neural Networks](https://mlsanity.com/paper/2009.06211/) (2020)
- [Learning Neural PDE Solvers with Convergence Guarantees](https://mlsanity.com/paper/1906.01200/) (2019)
- [Reviving and Improving Recurrent Back-Propagation](https://mlsanity.com/paper/1803.06396/) (2018)
- [Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction](https://mlsanity.com/paper/1905.10866/) (2019)
- [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018)
- [Anonymous Walk Embeddings](https://mlsanity.com/paper/1805.11921/) (2018)
- [Graph Element Networks: adaptive, structured computation and memory](https://mlsanity.com/paper/1904.09019/) (2019)
- [Rethinking Graph Regularization for Graph Neural Networks](https://mlsanity.com/paper/2009.02027/) (2020)
- [Fast and Deep Graph Neural Networks](https://mlsanity.com/paper/1911.08941/) (2019)
- [Hypersolvers: Toward Fast Continuous-Depth Models](https://mlsanity.com/paper/2007.09601/) (2020)
- [Learning Graph Structure With A Finite-State Automaton Layer](https://mlsanity.com/paper/2007.04929/) (2020)
- [Graph neural induction of value iteration](https://mlsanity.com/paper/2009.12604/) (2020)

## Built on by (most cited first, 12 of 12)

- [Deep Equilibrium Optical Flow Estimation](https://mlsanity.com/paper/2204.08442/) (2022)
- [Deep Equilibrium Approaches to Diffusion Models](https://mlsanity.com/paper/2210.12867/) (2022)
- [Optimization-Induced Graph Implicit Nonlinear Diffusion](https://mlsanity.com/paper/2206.14418/) (2022)
- [MGNNI: Multiscale Graph Neural Networks with Implicit Layers](https://mlsanity.com/paper/2210.08353/) (2022)
- [Learning context-aware adaptive solvers to accelerate quadratic programming](https://mlsanity.com/paper/2211.12443/) (2022)
- [IGNN-Solver: A Graph Neural Solver for Implicit Graph Neural Networks](https://mlsanity.com/paper/2410.08524/) (2024)
- [Bridging Graph Position Encodings for Transformers with Weighted Graph-Walking Automata](https://mlsanity.com/paper/2212.06898/) (2022)
- [Revisiting Generalized p-Laplacian Regularized Framelet GCNs: Convergence, Energy Dynamic and Training with Non-Linear Diffusion](https://mlsanity.com/paper/2305.15639/) (2023)
- [Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach](https://mlsanity.com/paper/2602.03297/) (2026)
- [Generalized Laplacian Regularized Framelet Graph Neural Networks](https://mlsanity.com/paper/2210.15092/) (2022)
- [Implicit Hypergraph Neural Network](https://mlsanity.com/paper/2508.14101/) (2025)
- [Multi-Attractor GNNs: Set-Valued Expressivity Beyond Unique Equilibria](https://mlsanity.com/paper/2609.35274/) (2026)

## Abstract (arXiv)

We propose the convergent graph solver (CGS), a deep learning method that learns iterative mappings to predict the properties of a graph system at its stationary state (fixed point) with guaranteed convergence. CGS systematically computes the fixed points of a target graph system and decodes them to estimate the stationary properties of the system without the prior knowledge of existing solvers or intermediate solutions. The forward propagation of CGS proceeds in three steps: (1) constructing the input dependent linear contracting iterative maps, (2) computing the fixed-points of the linear maps, and (3) decoding the fixed-points to estimate the properties. The contractivity of the constructed linear maps guarantees the existence and uniqueness of the fixed points following the Banach fixed point theorem. To train CGS efficiently, we also derive a tractable analytical expression for its gradient by leveraging the implicit function theorem. We evaluate the performance of CGS by applying it to various network-analytic and graph benchmark problems. The results indicate that CGS has competitive capabilities for predicting the stationary properties of graph systems, irrespective of whether the target systems are linear or non-linear. CGS also shows high performance for graph classification problems where the existence or the meaning of a fixed point is hard to be clearly defined, which highlights the potential of CGS as a general graph neural network architecture.