mlsanityData to 7 Oct 2026

Grappa: Gradient-Only Communication for Scalable Graph Neural Network Training

arXiv 2602.01872 ↗First posted 2 Feb 2026cs.DC cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Cross-partition edges dominate the cost of distributed GNN training: fetching remote features and activations per iteration overwhelms the network as graphs deepen and partition counts grow. Grappa is a distributed GNN training framework that enforces gradient-only communication: during each iteration, partitions train in isolation and exchange only gradients for the global update. To recover accuracy lost to isolation, Grappa (i) periodically repartitions to expose new neighborhoods and (ii) applies a lightweight coverage-corrected gradient aggregation inspired by importance sampling. We present an asymptotically unbiased estimator for gradient correction, which we use to develop a minimum-distance batch-level variant that is compatible with common deep-learning packages. We also introduce a shrinkage version that improves stability in practice. Empirical results on real and synthetic graphs show that Grappa trains GNNs 4x faster on average (up to 13x) than state-of-the-art systems, achieves better accuracy especially for deeper models, and sustains training at the trillion-edge scale on commodity hardware. Grappa is model-agnostic, supports full-graph and mini-batch training, and does not rely on high-bandwidth interconnects or caching.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 58 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

23 of 58 references.

References · 23

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  8. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  9. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  10. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  11. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
  12. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
11 more
  1. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 241
  2. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 152
  3. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  4. 2018Community Interaction and Conflict on the Webcited by 72
  5. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  6. 2022BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Samplingcited by 43
  7. 2020Accelerating Graph Sampling for Graph Machine Learning using GPUscited by 29
  8. 2023IGB: Addressing The Gaps In Labeling, Features, Heterogeneity, and Size of Public Graph Datasets for Deep Learning Researchcited by 29
  9. 2020Knowing your FATE: Friendship, Action and Temporal Explanations for User Engagement Prediction on Social Appscited by 23
  10. 2022MGG: Accelerating Graph Neural Networks with Fine-grained intra-kernel Communication-Computation Pipelining on Multi-GPU Platformscited by 16
  11. 2023Legion: Automatically Pushing the Envelope of Multi-GPU System for Billion-Scale GNN Trainingcited by 3

Built on by

Not cited yet.