# On the Feasibility of Simple Transformer for Dynamic Graph Modeling

arXiv: https://arxiv.org/abs/2401.14009 · HTML: https://mlsanity.com/paper/2401.14009/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2024.

## Built on by (most cited first, 22 of 22)

- [A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges](https://mlsanity.com/paper/2405.00476/) (2024)
- [Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking](https://mlsanity.com/paper/2410.01806/) (2024)
- [DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs](https://mlsanity.com/paper/2408.06966/) (2024)
- [TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics](https://mlsanity.com/paper/2502.02975/) (2025)
- [A Generalizable Anomaly Detection Method in Dynamic Graphs](https://mlsanity.com/paper/2412.16447/) (2024)
- [Retrieval Augmented Generation for Dynamic Graph Modeling](https://mlsanity.com/paper/2408.14523/) (2024)
- [Future Link Prediction Without Memory or Aggregation](https://mlsanity.com/paper/2505.19408/) (2025)
- [TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer](https://mlsanity.com/paper/2506.00431/) (2025)
- [Graph Memory Learning: Imitating Lifelong Remembering and Forgetting of Brain Networks](https://mlsanity.com/paper/2407.19183/) (2024)
- [FairGE: Fairness-Aware Graph Encoding in Incomplete Social Networks](https://mlsanity.com/paper/2601.09394/) (2026)
- [Unlocking Multi-Modal Potentials for Link Prediction on Dynamic Text-Attributed Graphs](https://mlsanity.com/paper/2502.19651/) (2025)
- [Temporal Graph Pattern Machine](https://mlsanity.com/paper/2601.22454/) (2026)
- [Graph Retention Networks for Dynamic Graphs](https://mlsanity.com/paper/2411.11259/) (2024)
- [ScaDyG:A New Paradigm for Large-scale Dynamic Graph Learning](https://mlsanity.com/paper/2501.16002/) (2025)
- [Invariant Graph Transformer for Out-of-Distribution Generalization](https://mlsanity.com/paper/2508.00304/) (2025)
- [TGTOD: A Global Temporal Graph Transformer for Outlier Detection at Scale](https://mlsanity.com/paper/2412.00984/) (2024)
- [Expressivity of Representation Learning on Continuous-Time Dynamic Graphs: An Information-Flow Centric Review](https://mlsanity.com/paper/2412.03783/) (2024)
- [Towards OOD Generalization in Dynamic Graphs via Causal Invariant Learning](https://mlsanity.com/paper/2603.01626/) (2026)
- [Kernelized Edge Attention: Addressing Semantic Attention Blurring in Temporal Graph Neural Networks](https://mlsanity.com/paper/2602.00596/) (2026)
- [Attention Dispersion in Dynamic Graph Transformers: Diagnosis and a Transferable Fix](https://mlsanity.com/paper/2605.16112/) (2026)
- [TGFormer: Towards Temporal Graph Transformer with Auto-Correlation Mechanism](https://mlsanity.com/paper/2605.24971/) (2026)
- [A Network-driven Framework for Public Event Forecasting via Dynamic Interaction Network Evolution](https://mlsanity.com/paper/2608.15488/) (2026)

## Abstract (arXiv)

Dynamic graph modeling is crucial for understanding complex structures in web graphs, spanning applications in social networks, recommender systems, and more. Most existing methods primarily emphasize structural dependencies and their temporal changes. However, these approaches often overlook detailed temporal aspects or struggle with long-term dependencies. Furthermore, many solutions overly complicate the process by emphasizing intricate module designs to capture dynamic evolutions. In this work, we harness the strength of the Transformer's self-attention mechanism, known for adeptly handling long-range dependencies in sequence modeling. Our approach offers a simple Transformer model, called SimpleDyG, tailored for dynamic graph modeling without complex modifications. We re-conceptualize dynamic graphs as a sequence modeling challenge and introduce a novel temporal alignment technique. This technique not only captures the inherent temporal evolution patterns within dynamic graphs but also streamlines the modeling process of their evolution. To evaluate the efficacy of SimpleDyG, we conduct extensive experiments on four real-world datasets from various domains. The results demonstrate the competitive performance of SimpleDyG in comparison to a series of state-of-the-art approaches despite its simple design.