mlsanityData to 7 Oct 2026

Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View

arXiv 1909.03211 ↗First posted 7 Sep 2019cs.LG cs.SI stat.MLcited by 544

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) have achieved promising performance on a wide range of graph-based tasks. Despite their success, one severe limitation of GNNs is the over-smoothing issue (indistinguishable representations of nodes in different classes). In this work, we present a systematic and quantitative study on the over-smoothing issue of GNNs. First, we introduce two quantitative metrics, MAD and MADGap, to measure the smoothness and over-smoothness of the graph nodes representations, respectively. Then, we verify that smoothing is the nature of GNNs and the critical factor leading to over-smoothness is the low information-to-noise ratio of the message received by the nodes, which is partially determined by the graph topology. Finally, we propose two methods to alleviate the over-smoothing issue from the topological view: (1) MADReg which adds a MADGap-based regularizer to the training objective;(2) AdaGraph which optimizes the graph topology based on the model predictions. Extensive experiments on 7 widely-used graph datasets with 10 typical GNN models show that the two proposed methods are effective for relieving the over-smoothing issue, thus improving the performance of various GNN models.

Key facts. Cited by 544 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 73; 2022: 110; 2023: 116; 2024: 117; 2025: 87; 2026: 40.

544 papers.

Sort:

By area:

  1. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 246
  2. 2023Language is All a Graph Needscited by 219
  3. 2022Representing Long-Range Context for Graph Neural Networks with Global Attentioncited by 193
  4. 2022Structure-Aware Transformer for Graph Representation Learningcited by 191
  5. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  6. 2021Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networkscited by 182
  7. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 166
  8. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  9. 2022Data Augmentation for Deep Graph Learning: A Surveycited by 132
  10. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  11. 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
  12. 2021An Empirical Study of Graph Contrastive Learningcited by 113
38 more
  1. 2021SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networkscited by 108
  2. 2021PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equationscited by 106
  3. 2022Not too little, not too much: a theoretical analysis of graph (over)smoothingcited by 106
  4. 2022FoSR: First-order spectral rewiring for addressing oversquashing in GNNscited by 104
  5. 2021Dirichlet Energy Constrained Learning for Deep Graph Neural Networkscited by 97
  6. 2022MultiScale MeshGraphNetscited by 94
  7. 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
  8. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  9. 2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 88
  10. 2021Topological Graph Neural Networkscited by 86
  11. 2022BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphscited by 80
  12. 2022DiffWire: Inductive Graph Rewiring via the Lov'asz Boundcited by 79
  13. 2023LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendationcited by 72
  14. 2023A Comprehensive Survey on Deep Graph Representation Learningcited by 72
  15. 2023FAENet: Frame Averaging Equivariant GNN for Materials Modelingcited by 72
  16. 2022Graph Rationalization with Environment-based Augmentationscited by 70
  17. 2024CARTE: Pretraining and Transfer for Tabular Learningcited by 69
  18. 2021FairDrop: Biased Edge Dropout for Enhancing Fairness in Graph Representation Learningcited by 68
  19. 2023Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothingcited by 67
  20. 2022Sheaf Neural Networks with Connection Laplacianscited by 64
  21. 2021Gophormer: Ego-Graph Transformer for Node Classificationcited by 63
  22. 2022Hypergraph Contrastive Collaborative Filteringcited by 63
  23. 2022On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networkscited by 63
  24. 2022Graph Data Augmentation for Graph Machine Learning: A Surveycited by 61
  25. 2021Topology-Imbalance Learning for Semi-Supervised Node Classificationcited by 56
  26. 2022Graph Filters for Signal Processing and Machine Learning on Graphscited by 53
  27. 2021Local Augmentation for Graph Neural Networkscited by 52
  28. 2022Gradient Gating for Deep Multi-Rate Learning on Graphscited by 49
  29. 2022A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networkscited by 49
  30. 2022MORE: Multi-Order RElation Mining for Dense Captioning in 3D Scenescited by 46
  31. 2021Vision Transformers with Patch Diversificationcited by 45
  32. 2023A Fractional Graph Laplacian Approach to Oversmoothingcited by 45
  33. 2021Simple GNN Regularisation for 3D Molecular Property Prediction & Beyondcited by 44
  34. 2021AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filtercited by 43
  35. 2022Efficient Learning of Mesh-Based Physical Simulation with BSMS-GNNcited by 43
  36. 2023Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networkscited by 43
  37. 2021Improving Graph Neural Networks with Simple Architecture Designcited by 41
  38. 2023Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All?cited by 41

50 most cited of 544 All 544 (JSON)