mlsanityData to 7 Oct 2026

Simplifying the Theory on Over-Smoothing

arXiv 2407.11876 ↗First posted 16 Jul 2024cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Graph convolutions have gained popularity due to their ability to efficiently operate on data with an irregular geometric structure. However, graph convolutions cause over-smoothing, which refers to representations becoming more similar with increased depth. However, many different definitions and intuitions currently coexist, leading to research efforts focusing on incompatible directions. This paper attempts to align these directions by showing that over-smoothing is merely a special case of power iteration. This greatly simplifies the existing theory on over-smoothing, making it more accessible. Based on the theory, we provide a novel comprehensive definition of rank collapse as a generalized form of over-smoothing and introduce the rank-one distance as a corresponding metric. Our empirical evaluation of 14 commonly used methods shows that more models than were previously known suffer from this issue.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

31 of 39 references.

References · 31

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2018How Powerful are Graph Neural Networks?cited by 4,355
  6. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  7. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
  8. 2020Simple and Deep Graph Convolutional Networkscited by 857
  9. 2015Gated Graph Sequence Neural Networkscited by 792
  10. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  11. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  12. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
19 more
  1. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 500
  2. 2021Beyond Low-frequency Information in Graph Convolutional Networkscited by 361
  3. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  4. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  5. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 245
  6. 2021Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networkscited by 182
  7. 2017Cooperative Game Theory Approaches for Network Partitioningcited by 134
  8. 2022Graph-Coupled Oscillator Networkscited by 123
  9. 2020Towards Deeper Graph Neural Networks with Differentiable Group Normalizationcited by 113
  10. 2020Implicit Graph Neural Networkscited by 98
  11. 2021Dirichlet Energy Constrained Learning for Deep Graph Neural Networkscited by 97
  12. 2022Understanding convolution on graphs via energiescited by 58
  13. 2023How does over-squashing affect the power of GNNs?cited by 53
  14. 2022Gradient Gating for Deep Multi-Rate Learning on Graphscited by 49
  15. 2023Demystifying Oversmoothing in Attention-Based Graph Neural Networkscited by 47
  16. 2023A Fractional Graph Laplacian Approach to Oversmoothingcited by 45
  17. 2024Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNscited by 31
  18. 2023Rank Collapse Causes Over-Smoothing and Over-Correlation in Graph Neural Networkscited by 18
  19. 2022Transforming PageRank into an Infinite-Depth Graph Neural Networkcited by 11

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Preventing Representational Rank Collapse in MPNNs by Splitting the Computational Graphcited by 6
  2. 2025Are We Measuring Oversmoothing in Graph Neural Networks Correctly?cited by 6
  3. 2024A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Neural Networkscited by 1
  4. 2025What Can We Learn From MIMO Graph Convolutions?cited by 1