# An Experimental Comparison of Partitioning Strategies for Distributed Graph Neural Network Training

arXiv: https://arxiv.org/abs/2308.15602 · HTML: https://mlsanity.com/paper/2308.15602/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 62 references that are in the map.
- First posted on arXiv 29 Aug 2023.

## Builds on (most cited first, 22 of 22)

- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs](https://mlsanity.com/paper/2010.05337/) (2020)
- [Improved Code Summarization via a Graph Neural Network](https://mlsanity.com/paper/2004.02843/) (2020)
- [Structure-aware Interactive Graph Neural Networks for the Prediction of Protein-Ligand Binding Affinity](https://mlsanity.com/paper/2107.10670/) (2021)
- [DistGNN: Scalable Distributed Training for Large-Scale Graph Neural Networks](https://mlsanity.com/paper/2104.06700/) (2021)
- [Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads](https://mlsanity.com/paper/2105.11118/) (2021)
- [Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis](https://mlsanity.com/paper/2205.09702/) (2022)
- [The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey](https://mlsanity.com/paper/2305.13854/) (2023)
- [More Recent Advances in (Hyper)Graph Partitioning](https://mlsanity.com/paper/2205.13202/) (2022)
- [Distributed Edge Partitioning for Trillion-edge Graphs](https://mlsanity.com/paper/1908.05855/) (2019)
- [Hybrid Edge Partitioner: Partitioning Large Power-Law Graphs under Memory Constraints](https://mlsanity.com/paper/2103.12594/) (2021)
- [ADWISE: Adaptive Window-based Streaming Edge Partitioning for High-Speed Graph Processing](https://mlsanity.com/paper/1712.08367/) (2017)
- [Scalable Edge Partitioning](https://mlsanity.com/paper/1808.06411/) (2018)
- [Out-of-Core Edge Partitioning at Linear Run-Time](https://mlsanity.com/paper/2203.12721/) (2022)
- [Spinner: Scalable Graph Partitioning in the Cloud](https://mlsanity.com/paper/1404.3861/) (2014)
- [Window-based Streaming Graph Partitioning Algorithm](https://mlsanity.com/paper/1902.01543/) (2019)
- [Partitioner Selection with EASE to Optimize Distributed Graph Processing](https://mlsanity.com/paper/2304.04976/) (2023)
- [BUbiNG: Massive Crawling for the Masses](https://mlsanity.com/paper/1601.06919/) (2016)
- [Partitioning Trillion-edge Graphs in Minutes](https://mlsanity.com/paper/1610.07220/) (2016)

## Built on by (most cited first, 4 of 4)

- [Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective](https://mlsanity.com/paper/2311.13279/) (2023)
- [Armada: Memory-Efficient Distributed Training of Large-Scale Graph Neural Networks](https://mlsanity.com/paper/2502.17846/) (2025)
- [Graph Learning Should Move Beyond Restrictive Views of Spectral and Message-Passing GNNs](https://mlsanity.com/paper/2602.10031/) (2026)
- [EmbedPart: Embedding-Driven Graph Partitioning for Scalable Graph Neural Network Training](https://mlsanity.com/paper/2604.01000/) (2026)

## Abstract (arXiv)

Recently, graph neural networks (GNNs) have gained much attention as a growing area of deep learning capable of learning on graph-structured data. However, the computational and memory requirements for training GNNs on large-scale graphs make it necessary to distribute the training. A prerequisite for distributed GNN training is to partition the input graph into smaller parts that are distributed among multiple machines of a compute cluster. Although graph partitioning has been studied with regard to graph analytics and graph databases, its effect on GNN training performance is largely unexplored. As a consequence, it is unclear whether investing computational efforts into high-quality graph partitioning would pay off in GNN training scenarios. In this paper, we study the effectiveness of graph partitioning for distributed GNN training. Our study aims to understand how different factors such as GNN parameters, mini-batch size, graph type, features size, and scale-out factor influence the effectiveness of graph partitioning. We conduct experiments with two different GNN systems using vertex and edge partitioning. We found that high-quality graph partitioning is a very effective optimization to speed up GNN training and to reduce memory consumption. Furthermore, our results show that invested partitioning time can quickly be amortized by reduced GNN training time, making it a relevant optimization for most GNN scenarios. Compared to research on distributed graph processing, our study reveals that graph partitioning plays an even more significant role in distributed GNN training, which motivates further research on the graph partitioning problem.