mlsanityData to 7 Oct 2026

Revisiting Over-smoothing in Deep GCNs

arXiv 2003.13663 ↗First posted 30 Mar 2020cs.LG stat.MLcited by 55

Not on the map. Why?

Abstract (arXiv)

Oversmoothing has been assumed to be the major cause of performance drop in deep graph convolutional networks (GCNs). In this paper, we propose a new view that deep GCNs can actually learn to anti-oversmooth during training. This work interprets a standard GCN architecture as layerwise integration of a Multi-layer Perceptron (MLP) and graph regularization. We analyze and conclude that before training, the final representation of a deep GCN does over-smooth, however, it learns anti-oversmoothing during training. Based on the conclusion, the paper further designs a cheap but effective trick to improve GCN training. We verify our conclusions and evaluate the trick on three citation networks and further provide insights on neighborhood aggregation in GCNs.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 14; 2023: 13; 2024: 7; 2025: 7; 2026: 4.

55 papers.

Sort:

By area:

  1. 2023Language is All a Graph Needscited by 219
  2. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  3. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  4. 2023On the Connection Between MPNN and Graph Transformercited by 60
  5. 2021On Provable Benefits of Depth in Training Graph Convolutional Networkscited by 58
  6. 2021Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoodscited by 57
  7. 2021Simple GNN Regularisation for 3D Molecular Property Prediction & Beyondcited by 44
  8. 2023Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphscited by 44
  9. 2021Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Studycited by 36
  10. 2022Transformers from an Optimization Perspectivecited by 27
  11. 2023Quantifying the Knowledge in GNNs for Reliable Distillation into MLPscited by 25
  12. 2022Learning to Sample and Aggregate: Few-shot Reasoning over Temporal Knowledge Graphscited by 21
38 more
  1. 2021Evaluating Deep Graph Neural Networkscited by 19
  2. 2022GATE: Graph CCA for Temporal SElf-supervised Learning for Label-efficient fMRI Analysiscited by 19
  3. 2021Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networkscited by 18
  4. 2025Oversmoothing, Oversquashing, Heterophily, Long-Range, and more: Demystifying Common Beliefs in Graph Machine Learningcited by 18
  5. 2023Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalizationcited by 12
  6. 2022RETE: Retrieval-Enhanced Temporal Event Forecasting on Unified Query Product Evolutionary Graphcited by 11
  7. 2022From Local to Global: Spectral-Inspired Graph Neural Networkscited by 11
  8. 2022Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Againcited by 11
  9. 2021SkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networkscited by 10
  10. 2024Local Action-Guided Motion Diffusion Model for Text-to-Motion Generationcited by 10
  11. 2024GRANOLA: Adaptive Normalization for Graph Neural Networkscited by 7
  12. 2021SSFG: Stochastically Scaling Features and Gradients for Regularizing Graph Convolutional Networkscited by 6
  13. 2023SGCN: Exploiting Compressed-Sparse Features in Deep Graph Convolutional Network Acceleratorscited by 4
  14. 2023Tokenized Graph Transformer with Neighborhood Augmentation for Node Classification in Large Graphscited by 4
  15. 2025Differential-Integral Neural Operator for Long-Term Turbulence Forecastingcited by 4
  16. 2023Principles for Initialization and Architecture Selection in Graph Neural Networks with ReLU Activationscited by 3
  17. 2024DTFormer: A Transformer-Based Method for Discrete-Time Dynamic Graph Representation Learningcited by 3
  18. 2022DOTIN: Dropping Task-Irrelevant Nodes for GNNscited by 2
  19. 2022Towards Understanding Graph Neural Networks: An Algorithm Unrolling Perspectivecited by 2
  20. 2022Application of Graph Neural Networks and graph descriptors for graph classificationcited by 2
  21. 2022Comprehensive Analysis of Over-smoothing in Graph Neural Networks from Markov Chains Perspectivecited by 2
  22. 2024Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspectivecited by 2
  23. 2024Graph Attention is Not Always Beneficial: A Theoretical Analysis of Graph Attention Mechanisms via Contextual Stochastic Block Modelscited by 2
  24. 2025Backward Oversmoothing: why is it hard to train deep Graph Neural Networks?cited by 2
  25. 2025M4GN: Mesh-based Multi-segment Hierarchical Graph Network for Dynamic Simulationscited by 2
  26. 2022NC-DRE: Leveraging Non-entity Clue Information for Document-level Relation Extractioncited by 1
  27. 2023A Model-Agnostic Graph Neural Network for Integrating Local and Global Informationcited by 1
  28. 2023Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNscited by 1
  29. 2024Mixture of Scope Experts at Test: Generalizing Deeper Graph Neural Networks with Shallow Variantscited by 1
  30. 2025Resolving Oversmoothing with Opinion Dissensuscited by 1
  31. 2025Anomaly Detection and Early Warning Mechanism for Intelligent Monitoring Systems in Multi-Cloud Environments Based on LLMcited by 1
  32. 2021Low-Rank Projections of GCNs Laplaciancited by 0
  33. 2021An FEA surrogate model with Boundary Oriented Graph Embedding approachcited by 0
  34. 2022Demystifying Graph Convolution with a Simple Concatenationcited by 0
  35. 2023Adaptive Depth Graph Attention Networkscited by 0
  36. 2023Graph Construction using Principal Axis Trees for Simple Graph Convolutioncited by 0
  37. 2023Supervised Attention Using Homophily in Graph Neural Networkscited by 0
  38. 2024HERTA: A High-Efficiency and Rigorous Training Algorithm for Unfolded Graph Neural Networkscited by 0

50 most cited of 55 All 55 (JSON)