mlsanityData to 14 Oct 2026

GNNBENCH: Fair and Productive Benchmarking for Single-GPU GNN System

arXiv 2404.04118 ↗First posted 5 Apr 2024cs.LG cs.DCcited by 3

Not on the map. Why?

Abstract (arXiv)

We hypothesize that the absence of a standardized benchmark has allowed several fundamental pitfalls in GNN System design and evaluation that the community has overlooked. In this work, we propose GNNBench, a plug-and-play benchmarking platform focused on system innovation. GNNBench presents a new protocol to exchange their captive tensor data, supports custom classes in System APIs, and allows automatic integration of the same system module to many deep learning frameworks, such as PyTorch and TensorFlow. To demonstrate the importance of such a benchmark framework, we integrated several GNN systems. Our results show that integration with GNNBench helped us identify several measurement issues that deserve attention from the community.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 27 of its 48 references that are in the map. First posted on arXiv 5 Apr 2024.

Builds on

27 of 48 references.

References · 27

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  7. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  8. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  9. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
  10. 2019Graph Neural Networks for Social Recommendationcited by 714
  11. 2017Residual Gated Graph ConvNetscited by 346
  12. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 242
15 more
  1. 2018Modeling polypharmacy side effects with graph convolutional networkscited by 237
  2. 2018GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphscited by 165
  3. 2020Sparse GPU Kernels for Deep Learningcited by 141
  4. 2018Embedding Logical Queries on Knowledge Graphscited by 120
  5. 2019MLPerf Training Benchmarkcited by 115
  6. 2020GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUscited by 51
  7. 2022SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learningcited by 47
  8. 2020GE-SpMM: General-purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networkscited by 42
  9. 2021Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspectivecited by 28
  10. 2020DOPS: Learning to Detect 3D Objects and Predict their 3D Shapescited by 27
  11. 2020Characterizing and Understanding GCNs on GPUcited by 22
  12. 2016Fathom: Reference Workloads for Modern Deep Learning Methodscited by 20
  13. 2021Efficient Graph Deep Learning in TensorFlow with tf_geometriccited by 17
  14. 2021TC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUscited by 17
  15. 2024Single-GPU GNN Systems: Traps and Pitfallscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024Graph Neural Networks Are More Than Filters: Revisiting and Benchmarking from A Spectral Perspectivecited by 3
  2. 2025GNN-Suite: a Graph Neural Network Benchmarking Framework for Biomedical Informaticscited by 2
  3. 2024Optimization of GNN Training Through Half-precisioncited by 1