mlsanityData to 7 Oct 2026

DeeperGCN: All You Need to Train Deeper GCNs

arXiv 2006.07739 ↗First posted 13 Jun 2020cs.LG cs.CV stat.MLcited by 249

Not on the map. Why?

Abstract (arXiv)

Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper. These challenges limit the representation power of GCNs on large-scale graphs. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs. We define differentiable generalized aggregation functions to unify different message aggregation operations (e.g. mean, max). We also propose a novel normalization layer namely MsgNorm and a pre-activation version of residual connections for GCNs. Extensive experiments on Open Graph Benchmark (OGB) show DeeperGCN significantly boosts performance over the state-of-the-art on the large scale graph learning tasks of node property prediction and graph property prediction. Please visit https://www.deepgcns.org for more information.

Key facts. Cited by 249 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 62; 2022: 51; 2023: 48; 2024: 51; 2025: 26; 2026: 11.

249 papers.

Sort:

By area:

  1. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 326
  2. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  3. 2023Language is All a Graph Needscited by 219
  4. 2022Structure-Aware Transformer for Graph Representation Learningcited by 191
  5. 2021Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networkscited by 176
  6. 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 172
  7. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  8. 2021Nested Graph Neural Networkscited by 136
  9. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  10. 2021A Unified Lottery Ticket Hypothesis for Graph Neural Networkscited by 118
  11. 2021Global Self-Attention as a Replacement for Graph Convolutioncited by 113
  12. 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 107
38 more
  1. 2021Dirichlet Energy Constrained Learning for Deep Graph Neural Networkscited by 97
  2. 2022Decoupling the Depth and Scope of Graph Neural Networkscited by 94
  3. 2023Where Did the Gap Go? Reassessing the Long-Range Graph Benchmarkcited by 83
  4. 2024Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classificationcited by 83
  5. 2022Graph Attention Multi-Layer Perceptroncited by 82
  6. 2022When Do Flat Minima Optimizers Work?cited by 78
  7. 2022One Transformer Can Understand Both 2D & 3D Molecular Datacited by 77
  8. 2022GRPE: Relative Positional Encoding for Graph Transformercited by 67
  9. 2023Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothingcited by 67
  10. 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modelingcited by 63
  11. 2021Graph Neural Networks Inspired by Classical Iterative Algorithmscited by 60
  12. 2021Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoodscited by 57
  13. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  14. 2022A Comprehensive Study on Large-Scale Graph Training: Benchmarking and Rethinkingcited by 56
  15. 2021Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depthcited by 55
  16. 2022Your Transformer May Not be as Powerful as You Expectcited by 55
  17. 2025PyG 2.0: Scalable Learning on Real World Graphscited by 49
  18. 2021Learnable Graph Matching: Incorporating Graph Partitioning with Deep Feature Learning for Multiple Object Trackingcited by 47
  19. 2021How Framelets Enhance Graph Neural Networkscited by 46
  20. 2021Simple GNN Regularisation for 3D Molecular Property Prediction & Beyondcited by 44
  21. 2021Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spottingcited by 43
  22. 2021Bag of Tricks for Node Classification with Graph Neural Networkscited by 40
  23. 2021Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networkscited by 38
  24. 2023Dink-Net: Neural Clustering on Large Graphscited by 38
  25. 2021Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Studycited by 36
  26. 2021MAAS: Multi-modal Assignation for Active Speaker Detectioncited by 35
  27. 2021Group-Aware Graph Neural Network for Nationwide City Air Quality Forecastingcited by 32
  28. 2024Graph Transformers: A Surveycited by 32
  29. 2021Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphscited by 31
  30. 2022MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initializationcited by 31
  31. 2021Variational Graph Normalized Auto-Encoderscited by 29
  32. 2021Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Ticketscited by 27
  33. 2021Dual-view Molecule Pre-trainingcited by 27
  34. 2021Scalable deeper graph neural networks for high-performance materials property predictioncited by 26
  35. 2021Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networkscited by 24
  36. 2021Directional Message Passing on Molecular Graphs via Synthetic Coordinatescited by 24
  37. 2021A New Perspective on the Effects of Spectrum in Graph Neural Networkscited by 24
  38. 2022Contrastive Laplacian Eigenmapscited by 24

50 most cited of 249 All 249 (JSON)