mlsanityData to 7 Oct 2026

Classifying Nodes in Graphs without GNNs

arXiv 2402.05934 ↗First posted 8 Feb 2024cs.LG cs.SIcited by 2

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) are the dominant paradigm for classifying nodes in a graph, but they have several undesirable attributes stemming from their message passing architecture. Recently, distillation methods succeeded in eliminating the use of GNNs at test time but they still require them during training. We perform a careful analysis of the role that GNNs play in distillation methods. This analysis leads us to propose a fully GNN-free approach for node classification, not requiring them at train or test time. Our method consists of three key components: smoothness constraints, pseudo-labeling iterations and neighborhood-label histograms. Our final approach can match the state-of-the-art accuracy on standard popular benchmarks such as citation and co-purchase networks, without training a GNN.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

30 of 41 references.

References · 30

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  9. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  10. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  11. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  12. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
18 more
  1. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  2. 2020Simple and Deep Graph Convolutional Networkscited by 857
  3. 2019Heterogeneous Graph Attention Networkcited by 732
  4. 2015Image-based Recommendations on Styles and Substitutescited by 702
  5. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  6. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  7. 2019Graph Transformer Networkscited by 520
  8. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  9. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 332
  10. 2014Learning with Pseudo-Ensemblescited by 241
  11. 2020Combining Label Propagation and Simple Models Out-performs Graph Neural Networkscited by 199
  12. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 151
  13. 2021Graph-MLP: Node Classification without Message Passing in Graphcited by 73
  14. 2021Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Frameworkcited by 69
  15. 2023Knowledge Distillation on Graphs: A Surveycited by 38
  16. 2019Graph-based Knowledge Distillation by Multi-head Attention Networkcited by 31
  17. 2022NOSMOG: Learning Noise-robust and Structure-aware MLPs on Graphscited by 14
  18. 2022Boosting Graph Neural Networks via Adaptive Knowledge Distillationcited by 14

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Graph Knowledge Distillation to Mixture of Expertscited by 2
  2. 2024Sparse Decomposition of Graph Neural Networkscited by 0