T-Graphormer: Using Transformers for Spatiotemporal Forecasting
Not on the map. Why?
Abstract (arXiv)
Spatiotemporal data is ubiquitous, and forecasting it has important applications in many domains. However, its complex cross-component dependencies and non-linear temporal dynamics can be challenging for traditional techniques. Existing methods address this by learning the two dimensions separately. Here, we introduce Temporal Graphormer (T-Graphormer), a Transformer-based approach capable of modelling spatiotemporal correlations simultaneously. By adding temporal encodings in the Graphormer architecture, each node attends to all other tokens within the graph sequence, enabling the model to learn rich spacetime patterns with minimal predefined inductive biases. We show the effectiveness of T-Graphormer on real-world traffic prediction benchmark datasets. Compared to state-of-the-art methods, T-Graphormer reduces root mean squared error (RMSE) and mean absolute percentage error (MAPE) by up to 20% and 10%.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 55 references that are in the map. First posted on arXiv 22 Jan 2025.
Builds on
31 of 55 references.
References · 31
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2016Layer Normalizationcited by 5,579
19 more
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
- 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
- 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 820
- 2020A Generalization of Transformer Networks to Graphscited by 653
- 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 563
- 2021Rethinking Graph Transformers with Spectral Attentioncited by 468
- 2019GMAN: A Graph Multi-Attention Network for Traffic Predictioncited by 433
- 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
- 2015Diffusion-Convolutional Neural Networkscited by 268
- 2022Vision GNN: An Image is Worth Graph of Nodescited by 185
- 2021Discrete Graph Structure Learning for Forecasting Multiple Time Seriescited by 169
- 2023PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Predictioncited by 135
- 2022Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecastingcited by 117
- 2023LargeST: A Benchmark Dataset for Large-Scale Traffic Forecastingcited by 116
- 2021Long-Range Transformers for Dynamic Spatiotemporal Forecastingcited by 46
Built on by
Citing papers per year: 2025: 2.
2 papers.