mlsanityData to 14 Oct 2026

Scalable Graph Neural Network Training: The Case for Sampling

arXiv 2105.02315 ↗First posted 5 May 2021cs.LG cs.DCcited by 11

Not on the map. Why?

Abstract (arXiv)

Graph Neural Networks (GNNs) are a new and increasingly popular family of deep neural network architectures to perform learning on graphs. Training them efficiently is challenging due to the irregular nature of graph data. The problem becomes even more challenging when scaling to large graphs that exceed the capacity of single devices. Standard approaches to distributed DNN training, such as data and model parallelism, do not directly apply to GNNs. Instead, two different approaches have emerged in the literature: whole-graph and sample-based training. In this paper, we review and compare the two approaches. Scalability is challenging with both approaches, but we make a case that research should focus on sample-based training since it is a more promising approach. Finally, we review recent systems supporting sample-based training.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 22 of its 44 references that are in the map. First posted on arXiv 5 May 2021.

Builds on

22 of 44 references.

References · 22

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Inductive Representation Learning on Large Graphscited by 6,320
  3. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  4. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  5. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  6. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
  7. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  8. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 584
  9. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 541
  10. 2014One weird trick for parallelizing convolutional neural networkscited by 264
  11. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 242
  12. 2018Adaptive Sampling Towards Fast Graph Representation Learningcited by 180
10 more
  1. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  2. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 153
  3. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  4. 2020Minimal Variance Sampling with Provable Guarantees for Fast Training of Graph Neural Networkscited by 54
  5. 2021Sampling methods for efficient training of graph convolutional networks: A surveycited by 40
  6. 2020Accelerating Graph Sampling for Graph Machine Learning using GPUscited by 29
  7. 2020Ripple Walk Training: A Subgraph-based training framework for Large and Deep Graph Neural Networkcited by 19
  8. 2020C-SAW: A Framework for Graph Sampling and Random Walk on GPUscited by 13
  9. 2020Peregrine: A Pattern-Aware Graph Mining Systemcited by 5
  10. 2015Arabesque: A System for Distributed Graph Mining - Extended versioncited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 6; 2024: 2; 2025: 1.

11 papers.

  1. 2022Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysiscited by 29
  2. 2023A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardwarecited by 18
  3. 2023The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Surveycited by 14
  4. 2023Social-LLM: Modeling User Behavior at Scale using Language Models and Social Network Datacited by 11
  5. 2024Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batchcited by 10
  6. 2022Scalable Graph Convolutional Network Training on Distributed-Memory Systemscited by 2
  7. 2023Distributed Graph Embedding with Information-Oriented Random Walkscited by 2
  8. 2023GraNNDis: Efficient Unified Distributed Training Framework for Deep GNNs on Large Clusterscited by 2
  9. 2023Towards a GML-Enabled Knowledge Graph Platformcited by 1
  10. 2024Reducing Memory Contention and I/O Congestion for Disk-based GNN Trainingcited by 1
  11. 2025Hierarchical graph sampling based minibatch learning with chain preservation and variance reductioncited by 0