mlsanityData to 14 Oct 2026

PairNorm: Tackling Oversmoothing in GNNs

arXiv 1909.12223 ↗First posted 26 Sep 2019cs.LG stat.MLcited by 297

Not on the map. Why?

Abstract (arXiv)

The performance of graph neural nets (GNNs) is known to gradually decrease with increasing number of layers. This decay is partly attributed to oversmoothing, where repeated graph convolutions eventually make node embeddings indistinguishable. We take a closer look at two different interpretations, aiming to quantify oversmoothing. Our main contribution is PairNorm, a novel normalization layer that is based on a careful analysis of the graph convolution operator, which prevents all node embeddings from becoming too similar. What is more, PairNorm is fast, easy to implement without any change to network architecture nor any additional parameters, and is broadly applicable to any GNN. Experiments on real-world graphs demonstrate that PairNorm makes deeper GCN, GAT, and SGC models more robust against oversmoothing, and significantly boosts performance for a new problem setting that benefits from deeper GNNs. Code is available at https://github.com/LingxiaoShawn/PairNorm.

Key facts. Cited by 297 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 54; 2022: 66; 2023: 56; 2024: 59; 2025: 32; 2026: 29.

297 papers.

Sort:

By area:

  1. 2021How Attentive are Graph Attention Networks?cited by 835
  2. 2021Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Predictioncited by 275
  3. 2023A Survey on Oversmoothing in Graph Neural Networkscited by 247
  4. 2022Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNscited by 214
  5. 2021Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networkscited by 182
  6. 2022Finding Global Homophily in Graph Neural Networks When Meeting Heterophilycited by 179
  7. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  8. 2021You are AllSet: A Multiset Function Framework for Hypergraph Neural Networkscited by 127
  9. 2022Pre-training via Denoising for Molecular Property Predictioncited by 124
  10. 2022Transformer for Graphs: An Overview from Architecture Perspectivecited by 107
  11. 2021PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equationscited by 106
  12. 2022Not too little, not too much: a theoretical analysis of graph (over)smoothingcited by 106
38 more
  1. 2022FoSR: First-order spectral rewiring for addressing oversquashing in GNNscited by 105
  2. 2022Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvaturecited by 98
  3. 2021Dirichlet Energy Constrained Learning for Deep Graph Neural Networkscited by 97
  4. 2022Decoupling the Depth and Scope of Graph Neural Networkscited by 94
  5. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  6. 2022Revisiting Over-smoothing in BERT from the Perspective of Graphcited by 72
  7. 2021Elastic Graph Neural Networkscited by 67
  8. 2022A Survey on Graph Representation Learning Methodscited by 66
  9. 2022Sheaf Neural Networks with Connection Laplacianscited by 65
  10. 2022On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networkscited by 64
  11. 2021Gophormer: Ego-Graph Transformer for Node Classificationcited by 63
  12. 2023On the Connection Between MPNN and Graph Transformercited by 61
  13. 2022Understanding convolution on graphs via energiescited by 59
  14. 2021Dissecting the Diffusion Process in Linear Graph Convolutional Networkscited by 58
  15. 2021On Provable Benefits of Depth in Training Graph Convolutional Networkscited by 58
  16. 2021Cold Brew: Distilling Graph Node Representations with Incomplete or Missing Neighborhoodscited by 58
  17. 2022GREAD: Graph Neural Reaction-Diffusion Networkscited by 54
  18. 2022Gradient Gating for Deep Multi-Rate Learning on Graphscited by 49
  19. 2022A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networkscited by 49
  20. 2023Demystifying Oversmoothing in Attention-Based Graph Neural Networkscited by 49
  21. 2021Search to aggregate neighborhood for graph neural networkcited by 48
  22. 2023A Fractional Graph Laplacian Approach to Oversmoothingcited by 45
  23. 2021Simple GNN Regularisation for 3D Molecular Property Prediction & Beyondcited by 44
  24. 2021AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filtercited by 43
  25. 2023Towards Understanding the Generalization of Graph Neural Networkscited by 41
  26. 2021Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networkscited by 39
  27. 2023ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyondcited by 39
  28. 2023Dink-Net: Neural Clustering on Large Graphscited by 38
  29. 2021Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Studycited by 36
  30. 2023Cooperative Graph Neural Networkscited by 36
  31. 2022FMP: Toward Fair Graph Message Passing against Topology Biascited by 35
  32. 2023HINormer: Representation Learning On Heterogeneous Information Networks with Graph Transformercited by 35
  33. 2024Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNscited by 34
  34. 2022EIGNN: Efficient Infinite-Depth Graph Neural Networkscited by 32
  35. 2022ACMP: Allen-Cahn Message Passing with Attractive and Repulsive Forces for Graph Neural Networkscited by 31
  36. 2021Heterogeneous Similarity Graph Neural Network on Electronic Health Recordscited by 30
  37. 2021Variational Graph Normalized Auto-Encoderscited by 29
  38. 2022DropMessage: Unifying Random Dropping for Graph Neural Networkscited by 29

50 most cited of 297 All 297 (JSON)