mlsanityData to 14 Oct 2026

BatchGNN: Efficient CPU-Based Distributed GNN Training on Very Large Graphs

arXiv 2306.13814 ↗First posted 23 Jun 2023cs.LG cs.DCcited by 2

Not on the map. Why?

Abstract (arXiv)

We present BatchGNN, a distributed CPU system that showcases techniques that can be used to efficiently train GNNs on terabyte-sized graphs. It reduces communication overhead with macrobatching in which multiple minibatches' subgraph sampling and feature fetching are batched into one communication relay to reduce redundant feature fetches when input features are static. BatchGNN provides integrated graph partitioning and native GNN layer implementations to improve runtime, and it can cache aggregated input features to further reduce sampling overhead. BatchGNN achieves an average 3× speedup over DistDGL on three GNN models trained on OGBN graphs, outperforms the runtimes reported by distributed GPU systems P3 and DistDGLv2, and scales to a terabyte-sized graph.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 37 references that are in the map. First posted on arXiv 23 Jun 2023.

Builds on

25 of 37 references.

References · 25

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2017Graph Attention Networkscited by 8,923
  4. 2017Inductive Representation Learning on Large Graphscited by 6,320
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  8. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  9. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 584
  10. 2020Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsterscited by 250
  11. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 242
  12. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
13 more
  1. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 153
  2. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  3. 2021GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddingscited by 90
  4. 2020Molecular graph generation with Graph Neural Networkscited by 80
  5. 2021DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networkscited by 59
  6. 2020Reducing Communication in Graph Neural Network Trainingcited by 50
  7. 2022BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Samplingcited by 43
  8. 2021Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipeliningcited by 39
  9. 2018Accurate, Efficient and Scalable Graph Embeddingcited by 35
  10. 2021Distributed Hybrid CPU and GPU training for Graph Neural Networks on Billion-Scale Graphscited by 33
  11. 2022Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Cachingcited by 21
  12. 2022Influence-Based Mini-Batching for Graph Neural Networkscited by 15
  13. 2020The LDBC Social Network Benchmarkcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024MassiveGNN: Efficient Training via Prefetching for Massively Connected Distributed Graphscited by 3
  2. 2025Enhanced Soups for Graph Neural Networkscited by 0