mlsanityData to 14 Oct 2026

BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing

arXiv 2112.08541 ↗First posted 16 Dec 2021cs.LG cs.DCcited by 50

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) have extended the success of deep neural networks (DNNs) to non-Euclidean graph data, achieving ground-breaking performance on various tasks such as node classification and graph property prediction. Nonetheless, existing systems are inefficient to train large graphs with billions of nodes and edges with GPUs. The main bottlenecks are the process of preparing data for GPUs - subgraph sampling and feature retrieving. This paper proposes BGL, a distributed GNN training system designed to address the bottlenecks with a few key ideas. First, we propose a dynamic cache engine to minimize feature retrieving traffic. By a co-design of caching policy and the order of sampling, we find a sweet spot of low overhead and high cache hit ratio. Second, we improve the graph partition algorithm to reduce cross-partition communication during subgraph sampling. Finally, careful resource isolation reduces contention between different data preprocessing stages. Extensive experiments on various GNN models and large graph datasets show that BGL significantly outperforms existing GNN training systems by 20.68x on average.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Builds on 30 of its 60 references that are in the map. First posted on arXiv 16 Dec 2021.

Builds on

30 of 60 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2017Graph Attention Networkscited by 8,923
  6. 2017Inductive Representation Learning on Large Graphscited by 6,320
  7. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  8. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  9. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  10. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  11. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  12. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
18 more
  1. 2018Link Prediction Based on Graph Neural Networkscited by 832
  2. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  3. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 584
  4. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 541
  5. 2017Representation Learning on Graphs: Methods and Applicationscited by 457
  6. 2018Deep Learning on Graphs: A Surveycited by 393
  7. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 242
  8. 2018Adaptive Sampling Towards Fast Graph Representation Learningcited by 180
  9. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 153
  10. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  11. 2017Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approachcited by 102
  12. 2020GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUscited by 51
  13. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  14. 2020Accelerating Graph Sampling for Graph Machine Learning using GPUscited by 29
  15. 2017Convergence Analysis of Distributed Stochastic Gradient Descent with Shufflingcited by 23
  16. 2020C-SAW: A Framework for Graph Sampling and Random Walk on GPUscited by 13
  17. 2019Distributed Edge Partitioning for Trillion-edge Graphscited by 7
  18. 2021An O(1) algorithm for implementing the LFU cache eviction schemecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 6; 2023: 15; 2024: 12; 2025: 10; 2026: 7.

50 papers.

Sort:

By area:

  1. 2022Trustworthy Graph Neural Networks: Aspects, Methods and Trendscited by 89
  2. 2022Distributed Graph Neural Network Training: A Surveycited by 32
  3. 2022Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysiscited by 29
  4. 2022Survey on Graph Neural Network Acceleration: An Algorithmic Perspectivecited by 26
  5. 2025HeuriGym: An Agentic Benchmark for LLM-Crafted Heuristics in Combinatorial Optimizationcited by 21
  6. 2022MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networkscited by 19
  7. 2023A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardwarecited by 18
  8. 2023The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Surveycited by 14
  9. 2023Slapo: A Schedule Language for Progressive Optimization of Large Deep Learning Model Trainingcited by 11
  10. 2023Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awarenesscited by 11
  11. 2023Adaptive Message Quantization and Parallelization for Distributed Full-graph GNN Trainingcited by 10
  12. 2024Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batchcited by 10
38 more
  1. 2023Comprehensive Evaluation of GNN Training Systems: A Data Management Perspectivecited by 9
  2. 2023SPEED: Streaming Partition and Parallel Acceleration for Temporal Interaction Graph Embeddingcited by 7
  3. 2023GSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelismcited by 6
  4. 2023GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphscited by 5
  5. 2024DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Trainingcited by 5
  6. 2023FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Trainingcited by 4
  7. 2023GNNPipe: Scaling Deep GNN Training with Pipelined Model Parallelismcited by 4
  8. 2024PyGim: An Efficient Graph Neural Network Library for Real Processing-In-Memory Architecturescited by 3
  9. 2024Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligencecited by 3
  10. 2024FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scalecited by 3
  11. 2024MassiveGNN: Efficient Training via Prefetching for Massively Connected Distributed Graphscited by 3
  12. 2022DGI: Easy and Efficient Inference for GNNscited by 2
  13. 2023GraNNDis: Efficient Unified Distributed Training Framework for Deep GNNs on Large Clusterscited by 2
  14. 2024GNNavigator: Towards Adaptive Training of Graph Neural Networks via Automatic Guideline Explorationcited by 2
  15. 2025Reforge: Low-Latency Distributed GNN Serving with Selective Embedding Recomputationcited by 2
  16. 2025Graph Learning at Scale: Characterizing and Optimizing Pre-Propagation GNNscited by 2
  17. 2025Efficient GNN Training Through Structure-Aware Randomized Mini-Batchingcited by 2
  18. 2025LPS-GNN : Deploying Graph Neural Networks on Graphs with 100-Billion Edgescited by 2
  19. 2023DGC: Training Dynamic Graphs with Spatio-Temporal Non-Uniformity using Graph Partitioning by Chunkscited by 1
  20. 2024GLISP: A Scalable GNN Learning System by Exploiting Inherent Structural Properties of Graphscited by 1
  21. 2024Single-GPU GNN Systems: Traps and Pitfallscited by 1
  22. 2024LSM-GNN: Large-scale Storage-based Multi-GPU GNN Training by Optimizing Data Transfer Schemecited by 1
  23. 2025Deal: Distributed End-to-End GNN Inference for All Nodescited by 1
  24. 2026Rudder: Steering Prefetching in Distributed GNN Training using LLM Agentscited by 1
  25. 2023Scalable Neural Network Training over Distributed Graphscited by 0
  26. 2023Synergistic Signals: Exploiting Co-Engagement and Semantic Links via Graph Neural Networkscited by 0
  27. 2024Towards providing reliable job completion time predictions using PCScited by 0
  28. 2024SDT-GNN: Streaming-based Distributed Training Framework for Graph Neural Networkscited by 0
  29. 2025Armada: Memory-Efficient Distributed Training of Large-Scale Graph Neural Networkscited by 0
  30. 2025Optimizing Federated Learning using Remote Embeddings for Graph Neural Networkscited by 0
  31. 2025Demystifying Distributed Training of Graph Neural Networks for Link Predictioncited by 0
  32. 2025AutoSAGE: Input-Aware CUDA Scheduling for Sparse GNN Aggregation (SpMM/SDDMM) and CSR Attentioncited by 0
  33. 2026Not All Neighbors Matter: Understanding the Impact of Graph Sparsification on GNN Pipelinescited by 0
  34. 2026Communication-free Sampling and 4D Hybrid Parallelism for Scalable Mini-batch GNN Trainingcited by 0
  35. 2026Scalable and Adaptive Parallel Training of Graph Transformer on Large Graphscited by 0
  36. 2026Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extendedcited by 0
  37. 2026FeLoG: Scalable and Efficient Distributed Graph Embedding with Feedback Loop Mechanismcited by 0
  38. 2026MegaGraph: Towards Efficient Training of Large-Scale Graph Transformers with Automated Hybrid Parallelismcited by 0