mlsanityData to 7 Oct 2026

GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training

arXiv 2009.03294 ↗First posted 7 Sep 2020cs.LG math.OC stat.MLcited by 116

Not on the map. Why?

Abstract (arXiv)

Normalization is known to help the optimization of deep neural networks. Curiously, different architectures require specialized normalization methods. In this paper, we study what normalization is effective for Graph Neural Networks (GNNs). First, we adapt and evaluate the existing methods from other domains to GNNs. Faster convergence is achieved with InstanceNorm compared to BatchNorm and LayerNorm. We provide an explanation by showing that InstanceNorm serves as a preconditioner for GNNs, but such preconditioning effect is weaker with BatchNorm due to the heavy batch noise in graph datasets. Second, we show that the shift operation in InstanceNorm results in an expressiveness degradation of GNNs for highly regular graphs. We address this issue by proposing GraphNorm with a learnable shift. Empirically, GNNs with GraphNorm converge faster compared to GNNs using other normalization. GraphNorm also improves the generalization of GNNs, achieving better performance on graph classification benchmarks.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 22; 2023: 14; 2024: 22; 2025: 17; 2026: 29.

116 papers.

Sort:

By area:

  1. 2021Do Transformers Really Perform Bad for Graph Representation?cited by 318
  2. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  3. 2024Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classificationcited by 83
  4. 2023FAENet: Frame Averaging Equivariant GNN for Materials Modelingcited by 72
  5. 2022GRPE: Relative Positional Encoding for Graph Transformercited by 67
  6. 2023Efficient 3D Semantic Segmentation with Superpoint Transformercited by 56
  7. 2021Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depthcited by 55
  8. 2022Raising the Bar in Graph-level Anomaly Detectioncited by 45
  9. 2021Accelerating Large Scale Real-Time GNN Inference using Channel Pruningcited by 41
  10. 2022Latent Graph Representations for Critical View of Safety Assessmentcited by 37
  11. 2024Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNscited by 31
  12. 2025Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physicscited by 28
38 more
  1. 2021A New Perspective on the Effects of Spectrum in Graph Neural Networkscited by 24
  2. 2025Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellencecited by 24
  3. 2022Molecular Representation Learning via Heterogeneous Motif Graph Neural Networkscited by 23
  4. 2023Neural incomplete factorization: learning preconditioners for the conjugate gradient methodcited by 18
  5. 2022Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Setscited by 17
  6. 2022Weisfeiler and Leman Go Infinite: Spectral and Combinatorial Pre-Coloringscited by 15
  7. 2022ResNorm: Tackling Long-tailed Degree Distribution Issue in Graph Neural Networks via Normalizationcited by 15
  8. 2022ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inferencecited by 15
  9. 2023MuSe-GNN: Learning Unified Gene Representation From Multimodal Biological Graph Datacited by 14
  10. 2023Efficiently predicting high resolution mass spectra with graph neural networkscited by 13
  11. 2022RSC: Accelerating Graph Neural Networks Training via Randomized Sparse Computationscited by 12
  12. 2022Gibbs-Helmholtz Graph Neural Network: capturing the temperature dependency of activity coefficients at infinite dilutioncited by 12
  13. 2023Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalizationcited by 12
  14. 2021Convergent Graph Solverscited by 11
  15. 2022From Local to Global: Spectral-Inspired Graph Neural Networkscited by 11
  16. 2023Connectivity Optimized Nested Graph Networks for Crystal Structurescited by 11
  17. 2024An end-to-end attention-based approach for learning on graphscited by 11
  18. 2022Graph Augmentation Learningcited by 9
  19. 2023Are GATs Out of Balance?cited by 9
  20. 2022Deep Transfer Learning with Graph Neural Network for Sensor-Based Human Activity Recognitioncited by 8
  21. 2022PCQA-GRAPHPOINT: Efficients Deep-Based Graph Metric For Point Cloud Quality Assessmentcited by 8
  22. 2024GRANOLA: Adaptive Normalization for Graph Neural Networkscited by 7
  23. 2021Calibrating and Improving Graph Contrastive Learningcited by 6
  24. 2024Logical Distillation of Graph Neural Networkscited by 6
  25. 2022BCNet: Bronchus Classification via Structure Guided Representation Learningcited by 5
  26. 2025Nonparametric Teaching for Graph Property Learnerscited by 5
  27. 2021On the Importance of Sampling in Training GCNs: Tighter Analysis and Variance Reductioncited by 4
  28. 2022FairNorm: Fair and Fast Graph Neural Network Trainingcited by 4
  29. 2022Metric Based Few-Shot Graph Classificationcited by 4
  30. 2024GATE: How to Keep Out Intrusive Neighborscited by 4
  31. 2024Enhancing Graph U-Nets for Mesh-Agnostic Spatio-Temporal Flow Predictioncited by 4
  32. 2024Contextualized Messages Boost Graph Representationscited by 3
  33. 2024Foundations and Frontiers of Graph Learning Theorycited by 3
  34. 2025Language-Guided Graph Representation Learning for Video Summarizationcited by 3
  35. 2022Towards Neural Sparse Linear Solverscited by 2
  36. 2022Graph Neural Network Expressivity and Meta-Learning for Molecular Property Regressioncited by 2
  37. 2024CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalizationcited by 2
  38. 2024MVGT: A Multi-view Graph Transformer Based on Spatial Relations for EEG Emotion Recognitioncited by 2

50 most cited of 116 All 116 (JSON)