mlsanityData to 14 Oct 2026

DistTGL: Distributed Memory-Based Temporal Graph Neural Network Training

arXiv 2307.07649 ↗First posted 14 Jul 2023cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Memory-based Temporal Graph Neural Networks are powerful tools in dynamic graph representation learning and have demonstrated superior performance in many real-world applications. However, their node memory favors smaller batch sizes to capture more dependencies in graph events and needs to be maintained synchronously across all trainers. As a result, existing frameworks suffer from accuracy loss when scaling to multiple GPUs. Evenworse, the tremendous overhead to synchronize the node memory make it impractical to be deployed to distributed GPU clusters. In this work, we propose DistTGL -- an efficient and scalable solution to train memory-based TGNNs on distributed GPU clusters. DistTGL has three improvements over existing solutions: an enhanced TGNN model, a novel training algorithm, and an optimized system. In experiments, DistTGL achieves near-linear convergence speedup, outperforming state-of-the-art single-machine method by 14.5% in accuracy and 10.17x in training throughput.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 14 Jul 2023.

Builds on

20 of 30 references.

References · 20

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  3. 2020Temporal Graph Networks for Deep Learning on Dynamic Graphscited by 530
  4. 2020Inductive Representation Learning on Temporal Graphscited by 462
  5. 2019Predicting Dynamic Embedding Trajectory in Temporal Interaction Networkscited by 448
  6. 2019EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphscited by 447
  7. 2021Inductive Representation Learning in Temporal Networks via Causal Anonymous Walkscited by 193
  8. 2017Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphscited by 149
  9. 2020DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphscited by 139
  10. 2022ROLAND: Graph Learning Framework for Dynamic Graphscited by 132
  11. 2022Towards Better Evaluation for Dynamic Link Predictioncited by 122
  12. 2019Variational Graph Recurrent Neural Networkscited by 87
8 more
  1. 2020APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embeddingcited by 78
  2. 2022TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphscited by 61
  3. 2021Distributed Hybrid CPU and GPU training for Graph Neural Networks on Billion-Scale Graphscited by 33
  4. 2022Instant Graph Neural Networks for Dynamic Graphscited by 16
  5. 2022Model-Architecture Co-Design for High Performance Temporal GNN Inference on FPGAcited by 12
  6. 2021Efficient Scaling of Dynamic Graph Neural Networkscited by 11
  7. 2018DynamicGEM: A Library for Dynamic Graph Embedding Methodscited by 8
  8. 2021Efficient Dynamic Graph Representation Learning at Scalecited by 8

Built on by

Cited by year

Citing papers per year: 2024: 9; 2025: 3.

12 papers.

  1. 2024A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challengescited by 30
  2. 2024A survey of dynamic graph neural networkscited by 24
  3. 2024Efficient Neural Common Neighbor for Temporal Graph Link Predictioncited by 18
  4. 2024From Link Prediction to Forecasting: Addressing Challenges in Batch-based Temporal Graph Learningcited by 3
  5. 2025PGT-I: Scaling Spatiotemporal GNNs with Memory-Efficient Distributed Trainingcited by 3
  6. 2025TGM: a Modular and Efficient Library for Machine Learning on Temporal Graphscited by 3
  7. 2024Scalable and Efficient Temporal Graph Representation Learning via Forward Recent Samplingcited by 2
  8. 2024TASER: Temporal Adaptive Sampling for Fast and Accurate Dynamic Graph Representation Learningcited by 2
  9. 2024Graph Retention Networks for Dynamic Graphscited by 2
  10. 2024Towards Ideal Temporal Graph Neural Networks: Evaluations and Conclusions after 10,000 GPU Hourscited by 2
  11. 2025TAMI: Taming Heterogeneity in Temporal Interactions for Temporal Graph Link Predictioncited by 2
  12. 2024Mind the truncation gap: challenges of learning on dynamic graphs with recurrent architecturescited by 1