mlsanityData to 7 Oct 2026

DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Training

arXiv 2405.05231 ↗First posted 8 May 2024cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) are machine learning models specialized for graph data and widely used in many applications. To train GNNs on large graphs that exceed CPU memory, several systems store data on disk and conduct out-of-core processing. However, these systems suffer from either read amplification when reading node features that are usually smaller than a disk page or degraded model accuracy by treating the graph as disconnected partitions. To close this gap, we build a system called DiskGNN, which achieves high I/O efficiency and thus fast training without hurting model accuracy. The key technique used by DiskGNN is offline sampling, which helps decouple graph sampling from model computation. In particular, by conducting graph sampling beforehand, DiskGNN acquires the node features that will be accessed by model computation, and such information is utilized to pack the target node features contiguously on disk to avoid read amplification. Besides, name also adopts designs including four-level feature store to fully utilize the memory hierarchy to cache node features and reduce disk access, batched packing to accelerate the feature packing process, and pipelined training to overlap disk access with other operations. We compare DiskGNN with Ginex and MariusGNN, which are state-of-the-art systems for out-of-core GNN training. The results show that DiskGNN can speed up the baselines by over 8x while matching their best model accuracy.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 41 of its 74 references that are in the map. First posted on arXiv 8 May 2024.

Builds on

41 of 74 references.

References · 41

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  7. 2018Link Prediction Based on Graph Neural Networkscited by 829
  8. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  9. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  10. 2020Graph Neural Networks in Recommender Systems: A Surveycited by 493
  11. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 241
  12. 2019Inductive Relation Prediction by Subgraph Reasoningcited by 237
29 more
  1. 2018Adaptive Sampling Towards Fast Graph Representation Learningcited by 180
  2. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  3. 2020Graph Clustering with Graph Neural Networkscited by 163
  4. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 159
  5. 2019Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensicscited by 158
  6. 2020A Semi-supervised Graph Attentive Network for Financial Fraud Detectioncited by 150
  7. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  8. 2018Billion-scale Commodity Embedding for E-commerce Recommendation in Alibabacited by 105
  9. 2018Temporal Relational Ranking for Stock Predictioncited by 100
  10. 2022Decoupling the Depth and Scope of Graph Neural Networkscited by 94
  11. 2020GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUscited by 51
  12. 2021BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessingcited by 50
  13. 2022PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communicationcited by 45
  14. 2022BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Samplingcited by 43
  15. 2017Hybrid Approach of Relation Network and Localized Graph Convolutional Filtering for Breast Cancer Subtype Classificationcited by 40
  16. 2018Accurate, Efficient and Scalable Graph Embeddingcited by 35
  17. 2020Bandit Samplers for Training Graph Neural Networkscited by 30
  18. 2021Marius: Learning Massive Graph Embeddings on a Single Machinecited by 29
  19. 2023IGB: Addressing The Gaps In Labeling, Features, Heterogeneity, and Size of Public Graph Datasets for Deep Learning Researchcited by 29
  20. 2021Reliable Graph Neural Networks for Drug Discovery Under Distributional Shiftcited by 22
  21. 2022Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Cachingcited by 21
  22. 2022MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networkscited by 19
  23. 2021QGTC: Accelerating Quantized Graph Neural Networks via GPU Tensor Corecited by 18
  24. 2021A Biased Graph Neural Network Sampler with Near-Optimal Regretcited by 17
  25. 2021TC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUscited by 17
  26. 2023Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awarenesscited by 11
  27. 2023Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accessescited by 6
  28. 2023Helios: An Efficient Out-of-core GNN Training System on Terabyte-scale Graphs with In-memory Performancecited by 4
  29. 2024Play like a Vertex: A Stackelberg Game Approach for Streaming Graph Partitioningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 3.

5 papers.

  1. 2024Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental Studycited by 4
  2. 2025OpenGLT: A Comprehensive Benchmark of Graph Neural Networks for Graph-Level Taskscited by 0
  3. 2026Not All Neighbors Matter: Understanding the Impact of Graph Sparsification on GNN Pipelinescited by 0
  4. 2026EmbedPart: Embedding-Driven Graph Partitioning for Scalable Graph Neural Network Trainingcited by 0
  5. 2026Taurus: Accelerating Out-of-Core Graph Neural Network Inference on Billion-Scale Graphscited by 0