# Towards Ideal Temporal Graph Neural Networks: Evaluations and Conclusions after 10,000 GPU Hours

arXiv: https://arxiv.org/abs/2412.20256 · HTML: https://mlsanity.com/paper/2412.20256/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 59 references that are in the map.
- First posted on arXiv 28 Dec 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Temporal Graph Networks for Deep Learning on Dynamic Graphs](https://mlsanity.com/paper/2006.10637/) (2020)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs](https://mlsanity.com/paper/1902.10191/) (2019)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)
- [Representation Learning for Dynamic Graphs: A Survey](https://mlsanity.com/paper/1905.11485/) (2019)
- [Time2Vec: Learning a Vector Representation of Time](https://mlsanity.com/paper/1907.05321/) (2019)
- [Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2012.09641/) (2020)
- [Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks](https://mlsanity.com/paper/2101.05974/) (2021)
- [ROLAND: Graph Learning Framework for Dynamic Graphs](https://mlsanity.com/paper/2208.07239/) (2022)
- [Towards Better Evaluation for Dynamic Link Prediction](https://mlsanity.com/paper/2207.10128/) (2022)
- [Sequential Recommendation with Graph Neural Networks](https://mlsanity.com/paper/2106.14226/) (2021)
- [APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embedding](https://mlsanity.com/paper/2011.11545/) (2020)
- [Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Space](https://mlsanity.com/paper/2107.03767/) (2021)
- [TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphs](https://mlsanity.com/paper/2203.14883/) (2022)
- [Structural Temporal Graph Neural Networks for Anomaly Detection in Dynamic Graphs](https://mlsanity.com/paper/2005.07427/) (2020)
- [Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning](https://mlsanity.com/paper/2202.13538/) (2022)
- [Deep learning for dynamic graphs: models and benchmarks](https://mlsanity.com/paper/2307.06104/) (2023)
- [Interaction-Based Trajectory Prediction Over a Hybrid Traffic Graph](https://mlsanity.com/paper/2009.12916/) (2020)
- [C-SAW: A Framework for Graph Sampling and Random Walk on GPUs](https://mlsanity.com/paper/2009.09103/) (2020)
- [DistTGL: Distributed Memory-Based Temporal Graph Neural Network Training](https://mlsanity.com/paper/2307.07649/) (2023)
- [Efficient Dynamic Graph Representation Learning at Scale](https://mlsanity.com/paper/2112.07768/) (2021)
- [CAT-Walk: Inductive Hypergraph Learning via Set Walks](https://mlsanity.com/paper/2306.11147/) (2023)
- [GraphAD: A Graph Neural Network for Entity-Wise Multivariate Time-Series Anomaly Detection](https://mlsanity.com/paper/2205.11139/) (2022)
- [DyG2Vec: Efficient Representation Learning for Dynamic Graphs](https://mlsanity.com/paper/2210.16906/) (2022)
- [GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs](https://mlsanity.com/paper/2311.17410/) (2023)
- [On the Generalization Capability of Temporal Graph Learning Algorithms: Theoretical Insights and a Simpler Method](https://mlsanity.com/paper/2402.16387/) (2024)

## Built on by (most cited first, 2 of 2)

- [Future Link Prediction Without Memory or Aggregation](https://mlsanity.com/paper/2505.19408/) (2025)
- [Back to All-Entity Ranking: Sampler-Dependent Evaluation in Continuous-Time Dynamic Graphs](https://mlsanity.com/paper/2607.27861/) (2026)

## Abstract (arXiv)

Temporal Graph Neural Networks (TGNNs) have emerged as powerful tools for modeling dynamic interactions across various domains. The design space of TGNNs is notably complex, given the unique challenges in runtime efficiency and scalability raised by the evolving nature of temporal graphs. We contend that many of the existing works on TGNN modeling inadequately explore the design space, leading to suboptimal designs. Viewing TGNN models through a performance-focused lens often obstructs a deeper understanding of the advantages and disadvantages of each technique. Specifically, benchmarking efforts inherently evaluate models in their original designs and implementations, resulting in unclear accuracy comparisons and misleading runtime. To address these shortcomings, we propose a practical comparative evaluation framework that performs a design space search across well-known TGNN modules based on a unified, optimized code implementation. Using our framework, we make the first efforts towards addressing three critical questions in TGNN design, spending over 10,000 GPU hours: (1) investigating the efficiency of TGNN module designs, (2) analyzing how the effectiveness of these modules correlates with dataset patterns, and (3) exploring the interplay between multiple modules. Key outcomes of this directed investigative approach include demonstrating that the most recent neighbor sampling and attention aggregator outperform uniform neighbor sampling and MLP-Mixer aggregator; Assessing static node memory as an effective node memory alternative, and showing that the choice between static or dynamic node memory should be based on the repetition patterns in the dataset. Our in-depth analysis of the interplay between TGNN modules and dataset patterns should provide a deeper insight into TGNN performance along with potential research directions for designing more general and effective TGNNs.