mlsanityData to 14 Oct 2026

ResNorm: Tackling Long-tailed Degree Distribution Issue in Graph Neural Networks via Normalization

arXiv 2206.08181 ↗First posted 16 Jun 2022cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) have attracted much attention due to their ability in learning representations from graph-structured data. Despite the successful applications of GNNs in many domains, the optimization of GNNs is less well studied, and the performance on node classification heavily suffers from the long-tailed node degree distribution. This paper focuses on improving the performance of GNNs via normalization. In detail, by studying the long-tailed distribution of node degrees in the graph, we propose a novel normalization method for GNNs, which is termed ResNorm (Reshaping the long-tailed distribution into a normal-like distribution via normalization). The scale operation of ResNorm reshapes the node-wise standard deviation (NStd) distribution so as to improve the accuracy of tail nodes (textiti.textite., low-degree nodes). We provide a theoretical interpretation and empirical evidence for understanding the mechanism of the above scale. In addition to the long-tailed distribution issue, over-smoothing is also a fundamental issue plaguing the community. To this end, we analyze the behavior of the standard shift and prove that the standard shift serves as a preconditioner on the weight matrix, increasing the risk of over-smoothing. With the over-smoothing issue in mind, we design a shift operation for ResNorm that simulates the degree-specific parameter strategy in a low-cost manner. Extensive experiments have validated the effectiveness of ResNorm on several node classification benchmark datasets.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 37 of its 63 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

37 of 63 references.

References · 37

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2016Layer Normalizationcited by 5,669
  6. 2018How Powerful are Graph Neural Networks?cited by 4,393
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  8. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  9. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  11. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
  12. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
25 more
  1. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 993
  2. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 877
  3. 2020Simple and Deep Graph Convolutional Networkscited by 860
  4. 2021How Attentive are Graph Attention Networks?cited by 835
  5. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
  6. 2019Graph Neural Networks for Social Recommendationcited by 714
  7. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 680
  8. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  9. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  10. 2020Towards Deeper Graph Neural Networkscited by 296
  11. 2021GraphDF: A Discrete Flow Model for Molecular Graph Generationcited by 180
  12. 2019MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendationcited by 121
  13. 2020GraphNorm: A Principled Approach to Accelerating Graph Neural Network Trainingcited by 117
  14. 2020Towards Deeper Graph Neural Networks with Differentiable Group Normalizationcited by 114
  15. 2016Range Loss for Deep Face Recognition with Long-tailcited by 111
  16. 2021Dirichlet Energy Constrained Learning for Deep Graph Neural Networkscited by 97
  17. 2021Few-Shot Graph Learning for Molecular Property Predictioncited by 81
  18. 2019DEMO-Net: Degree-specific Graph Neural Networks for Node and Graph Classificationcited by 59
  19. 2021Dissecting the Diffusion Process in Linear Graph Convolutional Networkscited by 58
  20. 2016Factors in Finetuning Deep Model for object detectioncited by 53
  21. 2020Understanding and Resolving Performance Degradation in Graph Convolutional Networkscited by 53
  22. 2020A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendationcited by 38
  23. 2018Addressing the Item Cold-start Problem by Attribute-driven Active Learningcited by 33
  24. 2018Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimizationcited by 27
  25. 2020MIRA: Leveraging Multi-Intention Co-click Information in Web-scale Document Retrieval using Deep Neural Networkscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 5; 2026: 3.

15 papers.

Sort:

By area:

  1. 2024How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspectivecited by 21
  2. 2024Advancing Graph Generation through Beta Diffusioncited by 13
  3. 2024Sign is Not a Remedy: Multiset-to-Multiset Message Passing for Learning on Heterophilic Graphscited by 8
  4. 2025Mitigating Over-Squashing in Graph Neural Networks by Spectrum-Preserving Sparsificationcited by 8
  5. 2024A Survey of Few-Shot Learning on Graphs: from Meta-Learning to Pre-Training and Prompt Learningcited by 6
  6. 2024Theoretical and Empirical Insights into the Origins of Degree Bias in Graph Neural Networkscited by 4
  7. 2024All-around Neural Collapse for Imbalanced Classificationcited by 3
  8. 2025Norm Augmented Graph AutoEncoders for Link Predictioncited by 2
  9. 2025Performance Heterogeneity in Graph Neural Networks: Lessons for Architecture Design and Preprocessingcited by 2
  10. 2023NP2L: Negative Pseudo Partial Labels Extraction for Graph Neural Networkscited by 0
  11. 2025Mitigating Degree Bias Adaptively with Hard-to-Learn Nodes in Graph Contrastive Learningcited by 0
  12. 2025Attributed Graph Clustering with Multi-Scale Weight-Based Pairwise Coarsening and Contrastive Learningcited by 0
3 more
  1. 2026Scalable Heterogeneous Graph Learning via Heterogeneous-aware Orthogonal Prototype Expertscited by 0
  2. 2026Placing Degree Scales After LayerNormcited by 0
  3. 2026Schreier-Coset Graph Rewiringcited by 0