mlsanityData to 7 Oct 2026

Taxonomy of Benchmarks in Graph Representation Learning

arXiv 2206.07729 ↗First posted 15 Jun 2022cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) extend the success of neural networks to graph-structured data by accounting for their intrinsic geometry. While extensive research has been done on developing GNN models with superior performance according to a collection of graph representation learning benchmarks, it is currently not well understood what aspects of a given model are probed by them. For example, to what extent do they test the ability of a model to leverage graph structure vs. node features? Here, we develop a principled approach to taxonomize benchmarking datasets according to a textitsensitivity profile that is based on how much GNN performance changes due to a collection of graph perturbations. Our data-driven analysis provides a deeper understanding of which benchmarking data characteristics are leveraged by GNNs. Consequently, our taxonomy can aid in selection and development of adequate graph benchmarks, and better informed evaluation of future GNN methods. Finally, our approach and implementation in textttGTaxoGym package are extendable to multiple graph prediction task types and future datasets.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 48 of its 72 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

48 of 72 references.

References · 48

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  8. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  9. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  10. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  11. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  12. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,571
36 more
  1. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  2. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  3. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  4. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  5. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  6. 2020Simple and Deep Graph Convolutional Networkscited by 857
  7. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  8. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
  9. 2020Benchmarking Graph Neural Networkscited by 672
  10. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  11. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  12. 2019Multi-scale Attributed Node Embeddingcited by 543
  13. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 471
  14. 2017Residual Gated Graph ConvNetscited by 341
  15. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 332
  16. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  17. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 309
  18. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 253
  19. 2020Combining Label Propagation and Simple Models Out-performs Graph Neural Networkscited by 199
  20. 2020The Surprising Power of Graph Neural Networks with Random Node Initializationcited by 190
  21. 2020Random Features Strengthen Graph Neural Networkscited by 184
  22. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  23. 2020Design Space for Graph Neural Networkscited by 182
  24. 2021Is Homophily a Necessity for Graph Neural Networks?cited by 182
  25. 2021Equivariant Subgraph Aggregation Networkscited by 154
  26. 2019A Survey on Graph Kernelscited by 148
  27. 2017Predicting multicellular function through multi-layer tissue networkscited by 138
  28. 2020Characteristic Functions on Graphs: Birds of a Feather, from Statistical Descriptors to Parametric Modelscited by 116
  29. 2021GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddingscited by 90
  30. 2020Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networkscited by 57
  31. 2022GraphWorld: Fake Graphs Bring Real Insights for GNNscited by 56
  32. 2020A Large-Scale Database for Graph Representation Learningcited by 54
  33. 2016Faster Kernels for Graphs with Continuous Attributes via Hashingcited by 31
  34. 2020Deep Graph Mapper: Seeing Graphs through the Neural Lenscited by 22
  35. 2019Coarse Graining of Data via Inhomogeneous Diffusion Condensationcited by 9
  36. 2021On Local Aggregation in Heterophilic Graphscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 4; 2025: 1; 2026: 1.

11 papers.

  1. 2022Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyondcited by 60
  2. 2023Graph Positional and Structural Encodercited by 31
  3. 2024MANTRA: The Manifold Triangulations Assemblagecited by 12
  4. 2023A Metadata-Driven Approach to Understand Graph Neural Networkscited by 10
  5. 2023Strengthening structural baselines for graph classification using Local Topological Profilecited by 7
  6. 2024Molecular Topological Profile (MOLTOP) -- Simple and Strong Baseline for Molecular Graph Classificationcited by 7
  7. 2022Graph Learning Indexer: A Contributor-Friendly and Metadata-Rich Platform for Graph Learning Benchmarkscited by 4
  8. 2024Towards Versatile Graph Learning Approach: from the Perspective of Large Language Modelscited by 1
  9. 2025Graph Alignment for Benchmarking Graph Neural Networks and Learning Positional Encodingscited by 1
  10. 2024Graph as a feature: improving node classification with non-neural graph-aware logistic regressioncited by 0
  11. 2026AdvSynGNN: Structure-Adaptive Graph Neural Nets via Adversarial Synthesis and Self-Corrective Propagationcited by 0