mlsanityData to 14 Oct 2026

Beyond Real-world Benchmark Datasets: An Empirical Study of Node Classification with GNNs

arXiv 2206.09144 ↗First posted 18 Jun 2022cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) have achieved great success on a node classification task. Despite the broad interest in developing and evaluating GNNs, they have been assessed with limited benchmark datasets. As a result, the existing evaluation of GNNs lacks fine-grained analysis from various characteristics of graphs. Motivated by this, we conduct extensive experiments with a synthetic graph generator that can generate graphs having controlled characteristics for fine-grained analysis. Our empirical studies clarify the strengths and weaknesses of GNNs from four major characteristics of real-world graphs with class labels of nodes, i.e., 1) class size distributions (balanced vs. imbalanced), 2) edge connection proportions between classes (homophilic vs. heterophilic), 3) attribute values (biased vs. random), and 4) graph sizes (small vs. large). In addition, to foster future research on GNNs, we publicly release our codebase that allows users to evaluate various GNNs with various graphs. We hope this work offers interesting insights for future research.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 18 Jun 2022.

Builds on

39 of 52 references.

References · 39

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2017Inductive Representation Learning on Large Graphscited by 6,320
  4. 2018How Powerful are Graph Neural Networks?cited by 4,393
  5. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  8. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  9. 2019Simplifying Graph Convolutional Networkscited by 1,477
  10. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  11. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 993
  12. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
27 more
  1. 2020Learning to Simulate Complex Physics with Graph Networkscited by 880
  2. 2018Representation Learning on Graphs with Jumping Knowledge Networkscited by 877
  3. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
  4. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 754
  5. 2020Benchmarking Graph Neural Networkscited by 680
  6. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  7. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 595
  8. 2020Heterogeneous Graph Transformercited by 582
  9. 2019Graph Transformer Networkscited by 523
  10. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  11. 2018Deep Learning on Graphs: A Surveycited by 393
  12. 2017Community Detection and Stochastic Block Modelscited by 319
  13. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 311
  14. 2020SIGN: Scalable Inception Graph Neural Networkscited by 284
  15. 2020Graph Neural Networks with Heterophilycited by 202
  16. 2021Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networkscited by 176
  17. 2022Graph Neural Networks for Graphs with Heterophily: A Surveycited by 175
  18. 2021GraphSMOTE: Imbalanced Node Classification on Graphs with Graph Neural Networkscited by 147
  19. 2018Contextual Stochastic Block Modelscited by 134
  20. 2019EEG-Based Emotion Recognition Using Regularized Graph Neural Networkscited by 97
  21. 2021New Benchmarks for Learning on Non-Homophilous Graphscited by 81
  22. 2022GraphWorld: Fake Graphs Bring Real Insights for GNNscited by 56
  23. 2021ImGAGN:Imbalanced Network Embedding via Generative Adversarial Graph Networkscited by 51
  24. 2021Improving Graph Neural Networks with Simple Architecture Designcited by 41
  25. 2020Network Embedding with Completely-imbalanced Labelscited by 21
  26. 2020Deep Graph Neural Networks with Shallow Subgraph Samplerscited by 16
  27. 2021GenCAT: Generating Attributed Graphs with Controlled Relationships between Classes, Attributes, and Topologycited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 12; 2025: 4; 2026: 2.

23 papers.

Sort:

By area:

  1. 2024Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classificationcited by 85
  2. 2024GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Datacited by 20
  3. 2024Mixture of Link Predictors on Graphscited by 10
  4. 2024A Comprehensive Benchmark on Spectral GNNs: The Impact on Efficiency, Memory, and Effectivenesscited by 6
  5. 2023Investigating the Interplay between Features and Structures in Graph Learningcited by 5
  6. 2023Bkd-FedGNN: A Benchmark for Classification Backdoor Attacks on Federated Graph Neural Networkcited by 4
  7. 2024Graph Neural Networks on Quantum Computerscited by 4
  8. 2024Revealing the Pitfalls and Re-Evaluating the Advancement of Heterophilic Graph Learningcited by 4
  9. 2025Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmarkcited by 4
  10. 2024Iterative Graph Neural Network Enhancement via Frequent Subgraph Mining of Explanationscited by 3
  11. 2023A Simple and Scalable Graph Neural Network for Large Directed Graphscited by 2
  12. 2024Unifews: You Need Fewer Operations for Efficient Graph Neural Networkscited by 2
11 more
  1. 2025Benchmarking Fairness-aware Graph Neural Networks in Knowledge Graphscited by 2
  2. 2026Structural Bias Beyond Homophily: A Study of Fairness in Link Predictioncited by 2
  3. 2023Towards Unsupervised Graph Completion Learning on Graphs with Features and Structure Missingcited by 1
  4. 2024MDS-GNN: A Mutual Dual-Stream Graph Neural Network on Graphs with Incomplete Features and Structurecited by 1
  5. 2025On Measuring Unnoticeability of Graph Adversarial Attacks: Observations, New Measure, and Applicationscited by 1
  6. 2023Explainable Graph Neural Architecture Search via Monte-Carlo Tree Search (Full version)cited by 0
  7. 2024Lying Graph Convolution: Learning to Lie for Node Classification Taskscited by 0
  8. 2024Hyperbolic Benchmarking Unveils Network Topology-Feature Relationship in GNN Performancecited by 0
  9. 2024Graph as a feature: improving node classification with non-neural graph-aware logistic regressioncited by 0
  10. 2025Predict, Cluster, Refine: A Joint Embedding Predictive Self-Supervised Framework for Graph Representation Learningcited by 0
  11. 2026Hyperbolic Graph Embedders for Link Prediction and Topology Reconstructioncited by 0