Plexus: Taming Billion-edge Graphs with 3D Parallel Full-graph GNN Training
Not on the map. Why?
Abstract (arXiv)
Graph neural networks (GNNs) leverage the connectivity and structure of real-world graphs to learn intricate properties and relationships between nodes. Many real-world graphs exceed the memory capacity of a GPU due to their sheer size, and training GNNs on such graphs requires techniques such as mini-batch sampling to scale. The alternative approach of distributed full-graph training suffers from high communication overheads and load imbalance due to the irregular structure of graphs. We propose a three-dimensional (3D) parallel approach for full-graph training that tackles these issues and scales to billion-edge graphs. In addition, we introduce optimizations such as a double permutation scheme for load balancing, and a performance model to predict the optimal 3D configuration of our parallel implementation -- Plexus. We evaluate Plexus on six different graph datasets and show scaling results on up to 2048 GPUs of Perlmutter, and 1024 GPUs of Frontier. Plexus achieves unprecedented speedups of 2.3-12.5x over prior state of the art, and a reduction in time-to-solution by 5.2-8.7x on Perlmutter and 7.0-54.2x on Frontier.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 52 references that are in the map. First posted on arXiv 7 May 2025.
Builds on
22 of 52 references.
References · 22
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
- 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,099
- 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
- 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
- 2021Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Trainingcited by 96
- 2022PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communicationcited by 45
- 2022BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Samplingcited by 43
- 2021Sampling methods for efficient training of graph convolutional networks: A surveycited by 40
10 more
- 2018Design Principles for Sparse Matrix Multiplication on the GPUcited by 18
- 2022MGG: Accelerating Graph Neural Networks with Fine-grained intra-kernel Communication-Computation Pipelining on Multi-GPU Platformscited by 16
- 2021AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learningcited by 15
- 2023Adaptive Message Quantization and Parallelization for Distributed Full-graph GNN Trainingcited by 10
- 2023Comprehensive Evaluation of GNN Training Systems: A Data Management Perspectivecited by 9
- 2025Democratizing AI: Open-source Scalable LLM Training on GPU-based Supercomputerscited by 7
- 2021MG-GCN: Scalable Multi-GPU GCN Training Frameworkcited by 4
- 2024NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelismcited by 4
- 2025Sparsity-Aware Communication for Distributed Graph Neural Network Trainingcited by 3
- 2024CDFGNN: a Systematic Design of Cache-based Distributed Full-Batch Graph Neural Network Training with Communication Reductioncited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.