mlsanityData to 7 Oct 2026

MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networks

arXiv 2202.02365 ↗First posted 4 Feb 2022cs.LG cs.DBcited by 19

Not on the map. Why?

Abstract (arXiv)

We study training of Graph Neural Networks (GNNs) for large-scale graphs. We revisit the premise of using distributed training for billion-scale graphs and show that for graphs that fit in main memory or the SSD of a single machine, out-of-core pipelined training with a single GPU can outperform state-of-the-art (SoTA) multi-GPU solutions. We introduce MariusGNN, the first system that utilizes the entire storage hierarchy -- including disk -- for GNN training. MariusGNN introduces a series of data organization and algorithmic contributions that 1) minimize the end-to-end time required for training and 2) ensure that models learned with disk-based training exhibit accuracy similar to those fully trained in memory. We evaluate MariusGNN against SoTA systems for learning GNN models and find that single-GPU training in MariusGNN achieves the same level of accuracy up to 8x faster than multi-GPU training in these systems, thus, introducing an order of magnitude monetary cost reduction. MariusGNN is open-sourced at www.marius-project.org.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 4 Feb 2022.

Builds on

34 of 60 references.

References · 34

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  5. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  7. 2014Embedding Entities and Relations for Learning and Inference in Knowledge Basescited by 929
  8. 2018Link Prediction Based on Graph Neural Networkscited by 829
  9. 2019Graph Neural Networks for Social Recommendationcited by 711
  10. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  11. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  12. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
22 more
  1. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  2. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 241
  3. 2019Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networkscited by 177
  4. 2021Discrete Graph Structure Learning for Forecasting Multiple Time Seriescited by 169
  5. 2020Machine Learning on Graphs: A Model and Comprehensive Taxonomycited by 156
  6. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 152
  7. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  8. 2019PyTorch-BigGraph: A Large-scale Graph Embedding Systemcited by 118
  9. 2020DGL-KE: Training Knowledge Graph Embeddings at Scalecited by 67
  10. 2018Random Shuffling Beats SGD after Finite Epochscited by 66
  11. 2020GNNAdvisor: An Adaptive and Efficient Runtime System for GNN Acceleration on GPUscited by 51
  12. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  13. 2021BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessingcited by 50
  14. 2019Estimating Node Importance in Knowledge Graphs Using Graph Neural Networkscited by 43
  15. 2015Variance Reduced Stochastic Gradient Descent with Neighborscited by 39
  16. 2021Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipeliningcited by 39
  17. 2021Distributed Hybrid CPU and GPU training for Graph Neural Networks on Billion-Scale Graphscited by 33
  18. 2020Accelerating Graph Sampling for Graph Machine Learning using GPUscited by 29
  19. 2021Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecturecited by 25
  20. 2021Global Neighbor Sampling for Mixed CPU-GPU Training on Giant Graphscited by 22
  21. 2021PyTorch-Direct: Enabling GPU Centric Data Access for Very Large Graph Neural Network Training with Irregular Accessescited by 12
  22. 2020GOSH: Embedding Big Graphs on Small Hardwarecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 5; 2025: 5; 2026: 3.

19 papers.

Sort:

By area:

  1. 2023High-Throughput Vector Similarity Search in Knowledge Graphscited by 19
  2. 2023A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardwarecited by 18
  3. 2023SemOpenAlex: The Scientific Landscape in 26 Billion RDF Triplescited by 12
  4. 2025Quake: Adaptive Indexing for Vector Searchcited by 8
  5. 2024DiskGNN: Bridging I/O Efficiency and Model Accuracy for Out-of-Core GNN Trainingcited by 5
  6. 2023FreshGNN: Reducing Memory Access via Stable Historical Embeddings for Graph Neural Network Trainingcited by 4
  7. 2023Growing and Serving Large Open-domain Knowledge Graphscited by 3
  8. 2024MassiveGNN: Efficient Training via Prefetching for Massively Connected Distributed Graphscited by 3
  9. 2023MuseGNN: Forming Scalable, Convergent GNN Layers that Minimize a Sampling-Based Energycited by 2
  10. 2025LPS-GNN : Deploying Graph Neural Networks on Graphs with 100-Billion Edgescited by 2
  11. 2024Single-GPU GNN Systems: Traps and Pitfallscited by 1
  12. 2024Reducing Memory Contention and I/O Congestion for Disk-based GNN Trainingcited by 1
7 more
  1. 2024LSM-GNN: Large-scale Storage-based Multi-GPU GNN Training by Optimizing Data Transfer Schemecited by 1
  2. 2025Armada: Memory-Efficient Distributed Training of Large-Scale Graph Neural Networkscited by 0
  3. 2025MLKV: Efficiently Scaling up Large Embedding Model Training with Disk-based Key-Value Storagecited by 0
  4. 2025AIRES: Accelerating Out-of-Core GCNs via Algorithm-System Co-Designcited by 0
  5. 2026Accelerating Storage-Based Training for Graph Neural Networkscited by 0
  6. 2026Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extendedcited by 0
  7. 2026Taurus: Accelerating Out-of-Core Graph Neural Network Inference on Billion-Scale Graphscited by 0