# TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphs

arXiv: https://arxiv.org/abs/2203.14883 · HTML: https://mlsanity.com/paper/2203.14883/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 28 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs](https://mlsanity.com/paper/1902.10191/) (2019)
- [OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs](https://mlsanity.com/paper/2103.09430/) (2021)
- [Representation Learning for Dynamic Graphs: A Survey](https://mlsanity.com/paper/1905.11485/) (2019)
- [Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks](https://mlsanity.com/paper/2101.05974/) (2021)
- [DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs](https://mlsanity.com/paper/2010.05337/) (2020)
- [PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models](https://mlsanity.com/paper/2104.07788/) (2021)
- [APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embedding](https://mlsanity.com/paper/2011.11545/) (2020)

## Built on by (most cited first, 30 of 61)

- [Towards Better Dynamic Graph Learning: New Architecture and Unified Library](https://mlsanity.com/paper/2303.13047/) (2023)
- [Do We Really Need Complicated Model Architectures For Temporal Networks?](https://mlsanity.com/paper/2302.11636/) (2023)
- [Temporal Graph Benchmark for Machine Learning on Temporal Graphs](https://mlsanity.com/paper/2307.01026/) (2023)
- [Graph Neural Networks for temporal graphs: State of the art, open challenges, and opportunities](https://mlsanity.com/paper/2302.01018/) (2023)
- [Neighborhood-aware Scalable Temporal Network Representation Learning](https://mlsanity.com/paper/2209.01084/) (2022)
- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs](https://mlsanity.com/paper/2406.12072/) (2024)
- [A survey of dynamic graph neural networks](https://mlsanity.com/paper/2404.18211/) (2024)
- [Spatio-Temporal Graph Neural Networks: A Survey](https://mlsanity.com/paper/2301.10569/) (2023)
- [A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardware](https://mlsanity.com/paper/2306.14052/) (2023)
- [TIGER: Temporal Interaction Graph Embedding with Restarts](https://mlsanity.com/paper/2302.06057/) (2023)
- [A Comprehensive Survey on Distributed Training of Graph Neural Networks](https://mlsanity.com/paper/2211.05368/) (2022)
- [Decoupled Graph Neural Networks for Large Dynamic Graphs](https://mlsanity.com/paper/2305.08273/) (2023)
- [Parameter-free Dynamic Graph Embedding for Link Prediction](https://mlsanity.com/paper/2210.08189/) (2022)
- [DistTGL: Distributed Memory-Based Temporal Graph Neural Network Training](https://mlsanity.com/paper/2307.07649/) (2023)
- [Disentangled Condensation for Large-scale Graphs](https://mlsanity.com/paper/2401.12231/) (2024)
- [Spear and Shield: Adversarial Attacks and Defense Methods for Model-Based Link Prediction on Continuous-Time Dynamic Graphs](https://mlsanity.com/paper/2308.10779/) (2023)
- [BenchTemp: A General Benchmark for Evaluating Temporal Graph Neural Networks](https://mlsanity.com/paper/2308.16385/) (2023)
- [SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning](https://mlsanity.com/paper/2303.03379/) (2023)
- [SPEED: Streaming Partition and Parallel Acceleration for Temporal Interaction Graph Embedding](https://mlsanity.com/paper/2308.14129/) (2023)
- [GraphStorm: all-in-one graph machine learning framework for industry applications](https://mlsanity.com/paper/2406.06022/) (2024)
- [NeutronStream: A Dynamic GNN Training Framework with Sliding Window for Graph Streams](https://mlsanity.com/paper/2312.02473/) (2023)
- [THeGCN: Temporal Heterophilic Graph Convolutional Network](https://mlsanity.com/paper/2412.16435/) (2024)
- [A Survey of Link Prediction in Temporal Networks](https://mlsanity.com/paper/2502.21185/) (2025)
- [Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models](https://mlsanity.com/paper/2503.14411/) (2025)
- [DyG2Vec: Efficient Representation Learning for Dynamic Graphs](https://mlsanity.com/paper/2210.16906/) (2022)
- [Efficient Dynamic Attributed Graph Generation](https://mlsanity.com/paper/2412.08810/) (2024)
- [PiPAD: Pipelined and Parallel Dynamic GNN Training on GPUs](https://mlsanity.com/paper/2301.00391/) (2023)
- [iLoRE: Dynamic Graph Representation with Instant Long-term Modeling and Re-occurrence Preservation](https://mlsanity.com/paper/2309.02012/) (2023)
- [DGNN-Booster: A Generic FPGA Accelerator Framework For Dynamic Graph Neural Network Inference](https://mlsanity.com/paper/2304.06831/) (2023)

## Abstract (arXiv)

Many real world graphs contain time domain information. Temporal Graph Neural Networks capture temporal information as well as structural and contextual information in the generated dynamic node embeddings. Researchers have shown that these embeddings achieve state-of-the-art performance in many different tasks. In this work, we propose TGL, a unified framework for large-scale offline Temporal Graph Neural Network training where users can compose various Temporal Graph Neural Networks with simple configuration files. TGL comprises five main components, a temporal sampler, a mailbox, a node memory module, a memory updater, and a message passing engine. We design a Temporal-CSR data structure and a parallel sampler to efficiently sample temporal neighbors to formtraining mini-batches. We propose a novel random chunk scheduling technique that mitigates the problem of obsolete node memory when training with a large batch size. To address the limitations of current TGNNs only being evaluated on small-scale datasets, we introduce two large-scale real-world datasets with 0.2 and 1.3 billion temporal edges. We evaluate the performance of TGL on four small-scale datasets with a single GPU and the two large datasets with multiple GPUs for both link prediction and node classification tasks. We compare TGL with the open-sourced code of five methods and show that TGL achieves similar or better accuracy with an average of 13x speedup. Our temporal parallel sampler achieves an average of 173x speedup on a multi-core CPU compared with the baselines. On a 4-GPU machine, TGL can train one epoch of more than one billion temporal edges within 1-10 hours. To the best of our knowledge, this is the first work that proposes a general framework for large-scale Temporal Graph Neural Networks training on multiple GPUs.