mlsanityData to 7 Oct 2026

DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs

arXiv 2010.05337 ↗First posted 11 Oct 2020cs.LG cs.DCcited by 139

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNN) have shown great success in learning from graph-structured data. They are widely used in various applications, such as recommendation, fraud detection, and search. In these domains, the graphs are typically large, containing hundreds of millions of nodes and several billions of edges. To tackle this challenge, we develop DistDGL, a system for training GNNs in a mini-batch fashion on a cluster of machines. DistDGL is based on the Deep Graph Library (DGL), a popular GNN development framework. DistDGL distributes the graph and its associated data (initial features and embeddings) across the machines and uses this distribution to derive a computational decomposition by following an owner-compute rule. DistDGL follows a synchronous training approach and allows ego-networks forming the mini-batches to include non-local nodes. To minimize the overheads associated with distributed computations, DistDGL uses a high-quality and light-weight min-cut graph partitioning algorithm along with multiple balancing constraints. This allows it to reduce communication overheads and statically balance the computations. It further reduces the communication by replicating halo nodes and by using sparse embedding updates. The combination of these design choices allows DistDGL to train high-quality models while achieving high parallel efficiency and memory scalability. We demonstrate our optimizations on both inductive and transductive GNN models. Our results show that DistDGL achieves linear speedup without compromising model accuracy and requires only 13 seconds to complete a training epoch for a graph with 100 million nodes and 3 billion edges on a cluster with 16 machines. DistDGL is now publicly available as part of DGL:https://github.com/dmlc/dgl/tree/master/python/dgl/distributed.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 30; 2023: 40; 2024: 23; 2025: 15; 2026: 12.

139 papers.

Sort:

By area:

  1. 2021Combinatorial Optimization with Physics-Inspired Graph Neural Networkscited by 107
  2. 2021GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddingscited by 90
  3. 2022Graph Attention Multi-Layer Perceptroncited by 82
  4. 2021DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networkscited by 59
  5. 2022TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphscited by 59
  6. 2021BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessingcited by 50
  7. 2022BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Samplingcited by 43
  8. 2021Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipeliningcited by 39
  9. 2022PaSca: a Graph Neural Architecture Search System under the Scalable Paradigmcited by 37
  10. 2022Distributed Graph Neural Network Training: A Surveycited by 32
  11. 2022Graph Coloring with Physics-Inspired Graph Neural Networkscited by 29
  12. 2022Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysiscited by 29
38 more
  1. 2023IGB: Addressing The Gaps In Labeling, Features, Heterogeneity, and Size of Public Graph Datasets for Deep Learning Researchcited by 29
  2. 2023Train Your Own GNN Teacher: Graph-Aware Distillation on Textual Graphscited by 29
  3. 2022Survey on Graph Neural Network Acceleration: An Algorithmic Perspectivecited by 26
  4. 2021Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecturecited by 25
  5. 2022Towards Training Billion Parameter Graph Neural Networks for Atomic Simulationscited by 25
  6. 2021Learn Locally, Correct Globally: A Distributed Algorithm for Training Graph Neural Networkscited by 24
  7. 2021Global Neighbor Sampling for Mixed CPU-GPU Training on Giant Graphscited by 22
  8. 2022TF-GNN: Graph Neural Networks in TensorFlowcited by 21
  9. 2022Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Cachingcited by 21
  10. 2021Evaluating Deep Graph Neural Networkscited by 19
  11. 2022MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networkscited by 19
  12. 2021Sequential Aggregation and Rematerialization: Distributed Full-batch Training of Graph Neural Networks on Large Graphscited by 18
  13. 2023A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardwarecited by 18
  14. 2023Communication-Efficient Graph Neural Networks with Probabilistic Neighborhood Expansion Analysis and Cachingcited by 16
  15. 2022A Comprehensive Survey on Distributed Training of Graph Neural Networkscited by 15
  16. 2021Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERTcited by 14
  17. 2023The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Surveycited by 13
  18. 2023Pitfalls in Link Prediction with Graph Neural Networks: Understanding the Impact of Target-link Inclusion & Better Practicescited by 13
  19. 2022SmartSAGE: Training Large-scale Graph Neural Networks using In-Storage Processing Architecturescited by 12
  20. 2022RSC: Accelerating Graph Neural Networks Training via Randomized Sparse Computationscited by 12
  21. 2023DistTGL: Distributed Memory-Based Temporal Graph Neural Network Trainingcited by 12
  22. 2023Graph Coarsening via Convolution Matching for Scalable Graph Neural Network Trainingcited by 12
  23. 2021Scalable Graph Neural Network Training: The Case for Samplingcited by 11
  24. 2022GNN at the Edge: Cost-Efficient Graph Neural Network Processing over Distributed Edge Serverscited by 11
  25. 2023Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awarenesscited by 11
  26. 2024Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batchcited by 10
  27. 2023Reply to: Modern graph neural networks do worse than classical greedy algorithms in solving combinatorial optimization problems like maximum independent setcited by 9
  28. 2021GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategycited by 8
  29. 2022Characterizing and Understanding Distributed GNN Training on GPUscited by 7
  30. 2023Simplifying Distributed Neural Network Training on Massive Graphs: Randomized Partitions Improve Model Aggregationcited by 7
  31. 2023SPEED: Streaming Partition and Parallel Acceleration for Temporal Interaction Graph Embeddingcited by 7
  32. 2024GraphStorm: all-in-one graph machine learning framework for industry applicationscited by 7
  33. 2022SUGAR: Efficient Subgraph-level Training via Resource-aware Graph Partitioningcited by 6
  34. 2022MultiBiSage: A Web-Scale Recommendation System Using Multiple Bipartite Graphs at Pinterestcited by 6
  35. 2022Nimble GNN Embedding with Tensor-Train Decompositioncited by 6
  36. 2023GSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelismcited by 6
  37. 2023Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accessescited by 6
  38. 2023Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Predictioncited by 6

50 most cited of 139 All 139 (JSON)