# Model Degradation Hinders Deep Graph Neural Networks

arXiv: https://arxiv.org/abs/2206.04361 · HTML: https://mlsanity.com/paper/2206.04361/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 57 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning](https://mlsanity.com/paper/1801.07606/) (2018)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Revisiting Semi-Supervised Learning with Graph Embeddings](https://mlsanity.com/paper/1603.08861/) (2016)
- [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018)
- [Representation Learning on Graphs with Jumping Knowledge Networks](https://mlsanity.com/paper/1806.03536/) (2018)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [DropEdge: Towards Deep Graph Convolutional Networks on Node Classification](https://mlsanity.com/paper/1907.10903/) (2019)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [Adaptive Universal Generalized PageRank Graph Neural Network](https://mlsanity.com/paper/2006.07988/) (2020)
- [Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View](https://mlsanity.com/paper/1909.03211/) (2019)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [Graph Neural Networks in Recommender Systems: A Survey](https://mlsanity.com/paper/2011.02260/) (2020)
- [Deep Learning on Graphs: A Survey](https://mlsanity.com/paper/1812.04202/) (2018)
- [PairNorm: Tackling Oversmoothing in GNNs](https://mlsanity.com/paper/1909.12223/) (2019)
- [Towards Deeper Graph Neural Networks](https://mlsanity.com/paper/2007.09296/) (2020)
- [SIGN: Scalable Inception Graph Neural Networks](https://mlsanity.com/paper/2004.11198/) (2020)
- [Revisiting Graph Neural Networks: All We Have is Low-Pass Filters](https://mlsanity.com/paper/1905.09550/) (2019)
- [A Note on Over-Smoothing for Graph Neural Networks](https://mlsanity.com/paper/2006.13318/) (2020)
- [Graph Random Neural Network for Semi-Supervised Learning on Graphs](https://mlsanity.com/paper/2005.11079/) (2020)
- [Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks](https://mlsanity.com/paper/2102.06462/) (2021)
- [Learning Human-Object Interactions by Graph Parsing Neural Networks](https://mlsanity.com/paper/1808.07962/) (2018)

## Built on by (most cited first, 26 of 26)

- [Learning Strong Graph Neural Networks with Weak Information](https://mlsanity.com/paper/2305.18457/) (2023)
- [Towards Deep Attention in Graph Neural Networks: Problems and Remedies](https://mlsanity.com/paper/2306.02376/) (2023)
- [Breaking the Entanglement of Homophily and Heterophily in Semi-supervised Node Classification](https://mlsanity.com/paper/2312.04111/) (2023)
- [Oversmoothing, Oversquashing, Heterophily, Long-Range, and more: Demystifying Common Beliefs in Graph Machine Learning](https://mlsanity.com/paper/2505.15547/) (2025)
- [Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach](https://mlsanity.com/paper/2506.13083/) (2025)
- [Decoupled Local Aggregation for Point Cloud Learning](https://mlsanity.com/paper/2308.16532/) (2023)
- [Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality Estimation](https://mlsanity.com/paper/2407.19944/) (2024)
- [Are GATs Out of Balance?](https://mlsanity.com/paper/2310.07235/) (2023)
- [Mitigating Oversmoothing Through Reverse Process of GNNs for Heterophilic Graphs](https://mlsanity.com/paper/2403.10543/) (2024)
- [Beyond Over-smoothing: Uncovering the Trainability Challenges in Deep Graph Neural Networks](https://mlsanity.com/paper/2408.03669/) (2024)
- [FreeGAD: A Training-Free yet Effective Approach for Graph Anomaly Detection](https://mlsanity.com/paper/2508.10594/) (2025)
- [Neighborhood Homophily-based Graph Convolutional Network](https://mlsanity.com/paper/2301.09851/) (2023)
- [Clenshaw Graph Neural Networks](https://mlsanity.com/paper/2210.16508/) (2022)
- [Clarify Confused Nodes via Separated Learning](https://mlsanity.com/paper/2306.02285/) (2023)
- [Mixture of Experts Meets Decoupled Message Passing: Towards General and Adaptive Node Classification](https://mlsanity.com/paper/2412.08193/) (2024)
- [IceBerg: Debiased Self-Training for Class-Imbalanced Node Classification](https://mlsanity.com/paper/2502.06280/) (2025)
- [Taming Gradient Oversmoothing and Expansion in Graph Neural Networks](https://mlsanity.com/paper/2410.04824/) (2024)
- [The Oversmoothing Fallacy: A Misguided Narrative in GNN Research](https://mlsanity.com/paper/2506.04653/) (2025)
- [Self-supervised Representation Learning on Electronic Health Records with Graph Kernel Infomax](https://mlsanity.com/paper/2209.00655/) (2022)
- [River of No Return: Graph Percolation Embeddings for Efficient Knowledge Graph Reasoning](https://mlsanity.com/paper/2305.09974/) (2023)
- [TSC: A Simple Two-Sided Constraint against Over-Smoothing](https://mlsanity.com/paper/2408.03152/) (2024)
- [Graph Wave Networks](https://mlsanity.com/paper/2505.20034/) (2025)
- [A Model-Agnostic Graph Neural Network for Integrating Local and Global Information](https://mlsanity.com/paper/2309.13459/) (2023)
- [Mixture of Scope Experts at Test: Generalizing Deeper Graph Neural Networks with Shallow Variants](https://mlsanity.com/paper/2409.06998/) (2024)
- [Multi-Label Node Classification with Label Influence Propagation](https://mlsanity.com/paper/2607.00671/) (2026)
- [GEN: A Practical Alternative to Graph Transformers for Long-Range Graph Modeling](https://mlsanity.com/paper/2401.01233/) (2024)

## Abstract (arXiv)

Graph Neural Networks (GNNs) have achieved great success in various graph mining tasks.However, drastic performance degradation is always observed when a GNN is stacked with many layers. As a result, most GNNs only have shallow architectures, which limits their expressive power and exploitation of deep neighborhoods.Most recent studies attribute the performance degradation of deep GNNs to the textitover-smoothing issue. In this paper, we disentangle the conventional graph convolution operation into two independent operations: textitPropagation (P) and textitTransformation (T).Following this, the depth of a GNN can be split into the propagation depth (Dp) and the transformation depth (Dt). Through extensive experiments, we find that the major cause for the performance degradation of deep GNNs is the textitmodel degradation issue caused by large Dt rather than the textitover-smoothing issue mainly caused by large Dp. Further, we present textitAdaptive Initial Residual (AIR), a plug-and-play module compatible with all kinds of GNN architectures, to alleviate the textitmodel degradation issue and the textitover-smoothing issue simultaneously. Experimental results on six real-world datasets demonstrate that GNNs equipped with AIR outperform most GNNs with shallow architectures owing to the benefits of both large Dp and Dt, while the time costs associated with AIR can be ignored.