# GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs

arXiv: https://arxiv.org/abs/2311.17410 · HTML: https://mlsanity.com/paper/2311.17410/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 71 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs](https://mlsanity.com/paper/1902.10191/) (2019)
- [AliGraph: A Comprehensive Graph Neural Network Platform](https://mlsanity.com/paper/1902.08730/) (2019)
- [Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs](https://mlsanity.com/paper/1705.05742/) (2017)
- [Streaming Graph Neural Networks](https://mlsanity.com/paper/1810.10627/) (2018)
- [Streaming Graph Neural Networks via Continual Learning](https://mlsanity.com/paper/2009.10951/) (2020)
- [BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing](https://mlsanity.com/paper/2112.08541/) (2021)
- [GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems](https://mlsanity.com/paper/2008.13517/) (2020)
- [Accelerating Graph Sampling for Graph Machine Learning using GPUs](https://mlsanity.com/paper/2009.06693/) (2020)
- [C-SAW: A Framework for Graph Sampling and Random Walk on GPUs](https://mlsanity.com/paper/2009.09103/) (2020)
- [Low-Latency Graph Streaming Using Compressed Purely-Functional Trees](https://mlsanity.com/paper/1904.08380/) (2019)
- [An O(1) algorithm for implementing the LFU cache eviction scheme](https://mlsanity.com/paper/2110.11602/) (2021)

## Built on by (most cited first, 5 of 5)

- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [MSPipe: Efficient Temporal GNN Training via Staleness-Aware Pipeline](https://mlsanity.com/paper/2402.15113/) (2024)
- [Towards Ideal Temporal Graph Neural Networks: Evaluations and Conclusions after 10,000 GPU Hours](https://mlsanity.com/paper/2412.20256/) (2024)
- [GNN-Suite: a Graph Neural Network Benchmarking Framework for Biomedical Informatics](https://mlsanity.com/paper/2505.10711/) (2025)
- [Directly Optimizing Mean Demographic Parity for Nonlinear Regression](https://mlsanity.com/paper/2607.05098/) (2026)

## Abstract (arXiv)

Graph Neural Networks (GNNs) play a crucial role in various fields. However, most existing deep graph learning frameworks assume pre-stored static graphs and do not support training on graph streams. In contrast, many real-world graphs are dynamic and contain time domain information. We introduce GNNFlow, a distributed framework that enables efficient continuous temporal graph representation learning on dynamic graphs on multi-GPU machines. GNNFlow introduces an adaptive time-indexed block-based data structure that effectively balances memory usage with graph update and sampling operation efficiency. It features a hybrid GPU-CPU graph data placement for rapid GPU-based temporal neighborhood sampling and kernel optimizations for enhanced sampling processes. A dynamic GPU cache for node and edge features is developed to maximize cache hit rates through reuse and restoration strategies. GNNFlow supports distributed training across multiple machines with static scheduling to ensure load balance. We implement GNNFlow based on DGL and PyTorch. Our experimental results show that GNNFlow provides up to 21.1x faster continuous learning than existing systems.