mlsanityData to 14 Oct 2026

Pitfalls of Graph Neural Network Evaluation

arXiv 1811.05868 ↗First posted 14 Nov 2018cs.LG cs.SI stat.MLcited by 1,007

Not on the map. Why?

Cited by 25 in Knowledge distillation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Semi-supervised node classification in graphs is a fundamental problem in graph mining, and the recently proposed graph neural networks (GNNs) have achieved unparalleled results on this task. Due to their massive success, GNNs have attracted a lot of attention, and many novel architectures have been put forward. In this paper we show that existing evaluation strategies for GNN models have serious shortcomings. We show that using the same train/validation/test splits of the same datasets, as well as making significant changes to the training procedure (e.g. early stopping criteria) precludes a fair comparison of different architectures. We perform a thorough empirical evaluation of four prominent GNN models and show that considering different splits of the data leads to dramatically different rankings of models. Even more importantly, our findings suggest that simpler GNN architectures are able to outperform the more sophisticated ones if the hyperparameters and the training procedure are tuned fairly for all models.

Key facts. Cited by 1,007 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 111; 2022: 163; 2023: 159; 2024: 235; 2025: 178; 2026: 161.

1,007 papers.

Sort:

By area:

  1. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 478
  2. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 343
  3. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 311
  4. 2021Graph Self-Supervised Learning: A Surveycited by 289
  5. 2021GRAND: Graph Neural Diffusioncited by 258
  6. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 243
  7. 2022Pure Transformers are Powerful Graph Learnerscited by 188
  8. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 186
  9. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  10. 2021Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networkscited by 182
  11. 2022Sign and Basis Invariant Networks for Spectral Graph Representation Learningcited by 179
  12. 2023Exphormer: Sparse Transformers for Graphscited by 170
38 more
  1. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  2. 2022How Powerful are Spectral Graph Neural Networkscited by 169
  3. 2022Handling Distribution Shifts on Graphs: An Invariance Perspectivecited by 164
  4. 2023All in One: Multi-task Prompting for Graph Neural Networkscited by 148
  5. 2021Hyperbolic Deep Neural Networks: A Surveycited by 139
  6. 2021Graph Unlearningcited by 138
  7. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  8. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  9. 2021Subgraph Federated Learning with Missing Neighbor Generationcited by 132
  10. 2022Graph-Coupled Oscillator Networkscited by 124
  11. 2022Towards Better Evaluation for Dynamic Link Predictioncited by 122
  12. 2022A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainabilitycited by 118
  13. 2021An Empirical Study of Graph Contrastive Learningcited by 113
  14. 2022How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervisioncited by 102
  15. 2021Robustness of Graph Neural Networks at Scalecited by 100
  16. 2021Dirichlet Energy Constrained Learning for Deep Graph Neural Networkscited by 97
  17. 2021Deep Graph Clustering via Dual Correlation Reductioncited by 97
  18. 2021GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddingscited by 90
  19. 2024All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretrainingcited by 86
  20. 2024Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classificationcited by 85
  21. 2021Beltrami Flow and Neural Diffusion on Graphscited by 84
  22. 2022Graph Attention Multi-Layer Perceptroncited by 83
  23. 2022What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoderscited by 82
  24. 2021New Benchmarks for Learning on Non-Homophilous Graphscited by 81
  25. 2021Scaling Up Graph Neural Networks Via Graph Coarseningcited by 80
  26. 2021Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classificationcited by 80
  27. 2022BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphscited by 80
  28. 2021Graph-based Semi-supervised Learning: A Comprehensive Reviewcited by 79
  29. 2021Multi-view Contrastive Graph Clusteringcited by 76
  30. 2021ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learningcited by 72
  31. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
  32. 2021Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learningcited by 70
  33. 2021Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Frameworkcited by 69
  34. 2021Subgroup Generalization and Fairness of Graph Neural Networkscited by 69
  35. 2022Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discriminationcited by 69
  36. 2023Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarkingcited by 69
  37. 2021Lorentzian Graph Convolutional Networkscited by 68
  38. 2021Elastic Graph Neural Networkscited by 67

50 most cited of 1,007 All 1,007 (JSON)