# Fast and Deep Graph Neural Networks

arXiv: https://arxiv.org/abs/1911.08941 · HTML: https://mlsanity.com/paper/1911.08941/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2019.

## Built on by (most cited first, 30 of 35)

- [ACMP: Allen-Cahn Message Passing with Attractive and Repulsive Forces for Graph Neural Networks](https://mlsanity.com/paper/2206.05437/) (2022)
- [Optimization-Induced Graph Implicit Nonlinear Diffusion](https://mlsanity.com/paper/2206.14418/) (2022)
- [VACA: Design of Variational Graph Autoencoders for Interventional and Counterfactual Queries](https://mlsanity.com/paper/2110.14690/) (2021)
- [MGNNI: Multiscale Graph Neural Networks with Implicit Layers](https://mlsanity.com/paper/2210.08353/) (2022)
- [Encoder-Decoder Architecture for Supervised Dynamic Graph Learning: A Survey](https://mlsanity.com/paper/2203.10480/) (2022)
- [Leave Graphs Alone: Addressing Over-Squashing without Rewiring](https://mlsanity.com/paper/2212.06538/) (2022)
- [Topology-aware Embedding Memory for Continual Learning on Expanding Networks](https://mlsanity.com/paper/2401.13200/) (2024)
- [Implicit vs Unfolded Graph Neural Networks](https://mlsanity.com/paper/2111.06592/) (2021)
- [Convergent Graph Solvers](https://mlsanity.com/paper/2106.01680/) (2021)
- [Transforming PageRank into an Infinite-Depth Graph Neural Network](https://mlsanity.com/paper/2207.00684/) (2022)
- [Pyramidal Reservoir Graph Neural Network](https://mlsanity.com/paper/2104.04710/) (2021)
- [Towards Quantum Graph Neural Networks: An Ego-Graph Learning Approach](https://mlsanity.com/paper/2201.05158/) (2022)
- [Continuous-Depth Neural Models for Dynamic Graph Prediction](https://mlsanity.com/paper/2106.11581/) (2021)
- [Weisfeiler and Leman go Hyperbolic: Learning Distance Preserving Node Representations](https://mlsanity.com/paper/2211.02501/) (2022)
- [Dynamic Graph Echo State Networks](https://mlsanity.com/paper/2110.08565/) (2021)
- [Few-shot Message-Enhanced Contrastive Learning for Graph Anomaly Detection](https://mlsanity.com/paper/2311.10370/) (2023)
- [Training-free Graph Neural Networks and the Power of Labels as Features](https://mlsanity.com/paper/2404.19288/) (2024)
- [Binary Graph Convolutional Network with Capacity Exploration](https://mlsanity.com/paper/2210.13149/) (2022)
- [What Do GNNs Actually Learn? Towards Understanding their Representations](https://mlsanity.com/paper/2304.10851/) (2023)
- [Addressing Heterophily in Node Classification with Graph Echo State Networks](https://mlsanity.com/paper/2305.08233/) (2023)
- [Edge of stability echo state networks](https://mlsanity.com/paper/2308.02902/) (2023)
- [Permute Me Softly: Learning Soft Permutations for Graph Representations](https://mlsanity.com/paper/2110.01872/) (2021)
- [IGNN-Solver: A Graph Neural Solver for Implicit Graph Neural Networks](https://mlsanity.com/paper/2410.08524/) (2024)
- [Beyond Homophily with Graph Echo State Networks](https://mlsanity.com/paper/2210.15731/) (2022)
- [An Empirical Evaluation of Rewiring Approaches in Graph Neural Networks](https://mlsanity.com/paper/2305.19717/) (2023)
- [Rapid training of Hamiltonian graph networks using random features](https://mlsanity.com/paper/2506.06558/) (2025)
- [Adaptive Initial Residual Connections for GNNs with Theoretical Guarantees](https://mlsanity.com/paper/2511.06598/) (2025)
- [Euler State Networks: Non-dissipative Reservoir Computing](https://mlsanity.com/paper/2203.09382/) (2022)
- [Modeling Edge Features with Deep Bayesian Graph Networks](https://mlsanity.com/paper/2308.09087/) (2023)
- [Efficient quantification on large-scale networks](https://mlsanity.com/paper/2503.15267/) (2025)

## Abstract (arXiv)

We address the efficiency issue for the construction of a deep graph neural network (GNN). The approach exploits the idea of representing each input graph as a fixed point of a dynamical system (implemented through a recurrent neural network), and leverages a deep architectural organization of the recurrent units. Efficiency is gained by many aspects, including the use of small and very sparse networks, where the weights of the recurrent units are left untrained under the stability condition introduced in this work. This can be viewed as a way to study the intrinsic power of the architecture of a deep GNN, and also to provide insights for the set-up of more complex fully-trained models. Through experimental results, we show that even without training of the recurrent connections, the architecture of small deep GNN is surprisingly able to achieve or improve the state-of-the-art performance on a significant set of tasks in the field of graphs classification.