mlsanityData to 7 Oct 2026

Residual connections provably mitigate oversmoothing in graph neural networks

arXiv 2501.00762 ↗First posted 1 Jan 2025cs.LG math.DS math.PRcited by 5

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing rates of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 1 Jan 2025.

Builds on

23 of 33 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  8. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
  9. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  10. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  11. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  12. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
11 more
  1. 2019Exact Combinatorial Optimization with Graph Convolutional Neural Networkscited by 313
  2. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  3. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 245
  4. 2022Not too little, not too much: a theoretical analysis of graph (over)smoothingcited by 106
  5. 2021A Review on Graph Neural Network Methods in Financial Applicationscited by 77
  6. 2021A Review of Graph Neural Networks and Their Applications in Power Systemscited by 62
  7. 2022A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networkscited by 49
  8. 2023Demystifying Oversmoothing in Attention-Based Graph Neural Networkscited by 47
  9. 2024Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNscited by 31
  10. 2022Graph Neural Networks Meet Wireless Communications: Motivation, Applications, and Future Directionscited by 8
  11. 2023Principles for Initialization and Architecture Selection in Graph Neural Networks with ReLU Activationscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Critical attention scaling in long-context transformerscited by 17
  2. 2025Backward Oversmoothing: why is it hard to train deep Graph Neural Networks?cited by 2
  3. 2025Adaptive Initial Residual Connections for GNNs with Theoretical Guaranteescited by 2
  4. 2026GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspectivecited by 0
  5. 2026Exploding and vanishing gradients in deep neural networks: the effect of residual connectionscited by 0