mlsanityData to 14 Oct 2026

Degree-based stratification of nodes in Graph Neural Networks

arXiv 2312.10458 ↗First posted 16 Dec 2023cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Despite much research, Graph Neural Networks (GNNs) still do not display the favorable scaling properties of other deep neural networks such as Convolutional Neural Networks and Transformers. Previous work has identified issues such as oversmoothing of the latent representation and have suggested solutions such as skip connections and sophisticated normalization schemes. Here, we propose a different approach that is based on a stratification of the graph nodes. We provide motivation that the nodes in a graph can be stratified into those with a low degree and those with a high degree and that the two groups are likely to behave differently. Based on this motivation, we modify the Graph Neural Network (GNN) architecture so that the weight matrices are learned, separately, for the nodes in each group. This simple-to-implement modification seems to improve performance across datasets and GNN methods. To verify that this increase in performance is not only due to the added capacity, we also perform the same modification for random splits of the nodes, which does not lead to any improvement.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 16 Dec 2023.

Builds on

24 of 29 references.

References · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  7. 2020Simple and Deep Graph Convolutional Networkscited by 860
  8. 2020A Generalization of Transformer Networks to Graphscited by 661
  9. 2020Heterogeneous Graph Transformercited by 582
  10. 2019Graph Transformer Networkscited by 523
  11. 2019Diffusion Improves Graph Learningcited by 414
  12. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 335
12 more
  1. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 333
  2. 2019PairNorm: Tackling Oversmoothing in GNNscited by 297
  3. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 222
  4. 2022How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervisioncited by 102
  5. 2019Attention Guided Graph Convolutional Networks for Relation Extractioncited by 82
  6. 2019DEMO-Net: Degree-specific Graph Neural Networks for Node and Graph Classificationcited by 59
  7. 2021Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depthcited by 55
  8. 2022A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networkscited by 49
  9. 2023ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyondcited by 39
  10. 2021A New Perspective on the Effects of Spectrum in Graph Neural Networkscited by 24
  11. 2022Exploiting Neighbor Effect: Conv-Agnostic GNNs Framework for Graphs with Heterophilycited by 15
  12. 2022Graph-adaptive Rectified Linear Unit for Graph Neural Networkscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compressioncited by 0