# PRES: Toward Scalable Memory-Based Dynamic Graph Neural Networks

arXiv: https://arxiv.org/abs/2402.04284 · HTML: https://mlsanity.com/paper/2402.04284/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2024.

## Built on by (most cited first, 10 of 10)

- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [Towards Robust Graph Incremental Learning on Evolving Graphs](https://mlsanity.com/paper/2402.12987/) (2024)
- [A Survey on Temporal Interaction Graph Representation Learning: Progress, Challenges, and Opportunities](https://mlsanity.com/paper/2505.04461/) (2025)
- [On the Topology Awareness and Generalization Performance of Graph Neural Networks](https://mlsanity.com/paper/2403.04482/) (2024)
- [From Link Prediction to Forecasting: Addressing Challenges in Batch-based Temporal Graph Learning](https://mlsanity.com/paper/2406.04897/) (2024)
- [Expressivity of Representation Learning on Continuous-Time Dynamic Graphs: An Information-Flow Centric Review](https://mlsanity.com/paper/2412.03783/) (2024)
- [Decoupled and Divergence-Conditioned Prompt for Multi-domain Dynamic Graph Foundation Models](https://mlsanity.com/paper/2605.13540/) (2026)
- [Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach](https://mlsanity.com/paper/2608.15224/) (2026)
- [Attention Dispersion in Dynamic Graph Transformers: Diagnosis and a Transferable Fix](https://mlsanity.com/paper/2605.16112/) (2026)
- [Structuring Relations Among Learning Paradigms via Protocol--Objective--Resource Reductions](https://mlsanity.com/paper/2609.32766/) (2026)

## Abstract (arXiv)

Memory-based Dynamic Graph Neural Networks (MDGNNs) are a family of dynamic graph neural networks that leverage a memory module to extract, distill, and memorize long-term temporal dependencies, leading to superior performance compared to memory-less counterparts. However, training MDGNNs faces the challenge of handling entangled temporal and structural dependencies, requiring sequential and chronological processing of data sequences to capture accurate temporal patterns. During the batch training, the temporal data points within the same batch will be processed in parallel, while their temporal dependencies are neglected. This issue is referred to as temporal discontinuity and restricts the effective temporal batch size, limiting data parallelism and reducing MDGNNs' flexibility in industrial applications. This paper studies the efficient training of MDGNNs at scale, focusing on the temporal discontinuity in training MDGNNs with large temporal batch sizes. We first conduct a theoretical study on the impact of temporal batch size on the convergence of MDGNN training. Based on the analysis, we propose PRES, an iterative prediction-correction scheme combined with a memory coherence learning objective to mitigate the effect of temporal discontinuity, enabling MDGNNs to be trained with significantly larger temporal batches without sacrificing generalization performance. Experimental results demonstrate that our approach enables up to a 4x larger temporal batch (3.4x speed-up) during MDGNN training.