mlsanityData to 7 Oct 2026

Efficient Scaling of Dynamic Graph Neural Networks

arXiv 2109.07893 ↗First posted 16 Sep 2021cs.DC cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

We present distributed algorithms for training dynamic Graph Neural Networks (GNN) on large scale graphs spanning multi-node, multi-GPU systems. To the best of our knowledge, this is the first scaling study on dynamic GNN. We devise mechanisms for reducing the GPU memory usage and identify two execution time bottlenecks: CPU-GPU data transfer; and communication volume. Exploiting properties of dynamic graphs, we design a graph difference-based strategy to significantly reduce the transfer time. We develop a simple, but effective data distribution technique under which the communication volume remains fixed and linear in the input size, for any number of GPUs. Our experiments using billion-size graphs on a system of 128 GPUs shows that: (i) the distribution scheme achieves up to 30x speedup on 128 GPUs; (ii) the graph-difference technique reduces the transfer time by a factor of up to 4.1x and the overall execution time by up to 40%

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 16 Sep 2021.

Builds on

19 of 31 references.

References · 19

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  3. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  4. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  5. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  6. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 521
  7. 2019EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphscited by 443
  8. 2016Structured Sequence Modeling with Graph Convolutional Recurrent Networkscited by 273
  9. 2019Representation Learning for Dynamic Graphs: A Surveycited by 260
  10. 2019Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networkscited by 241
  11. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 152
  12. 2017Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphscited by 149
7 more
  1. 2018Streaming Graph Neural Networkscited by 91
  2. 2017Dynamic Graph Convolutional Networkscited by 83
  3. 2020Computing Graph Neural Networks: A Survey from Algorithms to Acceleratorscited by 81
  4. 2016Memory-Efficient Backpropagation Through Timecited by 78
  5. 2020Reducing Communication in Graph Neural Network Trainingcited by 50
  6. 2018Scalable Graph Learning for Anti-Money Laundering: A First Lookcited by 32
  7. 2018Learning Dynamic Embeddings from Temporal Interactionscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 3; 2025: 0; 2026: 1.

11 papers.

  1. 2023Realistic Synthetic Financial Transactions for Anti-Money Laundering Modelscited by 47
  2. 2022Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysiscited by 29
  3. 2024A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challengescited by 29
  4. 2024A survey of dynamic graph neural networkscited by 21
  5. 2022A Comprehensive Survey on Distributed Training of Graph Neural Networkscited by 15
  6. 2023DistTGL: Distributed Memory-Based Temporal Graph Neural Network Trainingcited by 12
  7. 2023PiPAD: Pipelined and Parallel Dynamic GNN Training on GPUscited by 4
  8. 2023DGNN-Booster: A Generic FPGA Accelerator Framework For Dynamic Graph Neural Network Inferencecited by 3
  9. 2024Input Snapshots Fusion for Scalable Discrete-Time Dynamic Graph Neural Networkscited by 3
  10. 2023DGC: Training Dynamic Graphs with Spatio-Temporal Non-Uniformity using Graph Partitioning by Chunkscited by 1
  11. 2026DG-CoLearn: An Efficient Collaborative Learning Framework for Dynamic Graphscited by 0