# STAEformer: Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting

arXiv: https://arxiv.org/abs/2308.10425 · HTML: https://mlsanity.com/paper/2308.10425/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 21 Aug 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [GMAN: A Graph Multi-Attention Network for Traffic Prediction](https://mlsanity.com/paper/1911.08415/) (2019)
- [T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction](https://mlsanity.com/paper/1811.05320/) (2018)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)
- [Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2012.09641/) (2020)
- [Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2106.12931/) (2021)
- [Spatial-Temporal Transformer Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2001.02908/) (2020)
- [Discrete Graph Structure Learning for Forecasting Multiple Time Series](https://mlsanity.com/paper/2101.06861/) (2021)
- [PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction](https://mlsanity.com/paper/2301.07945/) (2023)
- [Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecasting](https://mlsanity.com/paper/2206.09112/) (2022)
- [Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2206.09113/) (2022)
- [Spatio-Temporal Meta-Graph Learning for Traffic Forecasting](https://mlsanity.com/paper/2211.14701/) (2022)
- [DL-Traff: Survey and Benchmark of Deep Learning Models for Urban Traffic Prediction](https://mlsanity.com/paper/2108.09091/) (2021)
- [Z-GCNETs: Time Zigzags at Graph Convolutional Networks for Time Series Forecasting](https://mlsanity.com/paper/2105.04100/) (2021)
- [Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Version](https://mlsanity.com/paper/2203.15737/) (2022)
- [Coupled Layer-wise Graph Convolution for Transportation Demand Prediction](https://mlsanity.com/paper/2012.08080/) (2020)
- [Historical Inertia: A Neglected but Powerful Baseline for Long Sequence Time-series Forecasting](https://mlsanity.com/paper/2103.16349/) (2021)
- [A Multi-view Multi-task Learning Framework for Multi-variate Time Series Forecasting](https://mlsanity.com/paper/2109.01657/) (2021)
- [Learning to Remember Patterns: Pattern Matching Memory Networks for Traffic Forecasting](https://mlsanity.com/paper/2110.10380/) (2021)
- [Event-Aware Multimodal Mobility Nowcasting](https://mlsanity.com/paper/2112.08443/) (2021)
- [Auto-STGCN: Autonomous Spatial-Temporal Graph Convolutional Network Search Based on Reinforcement Learning and Existing Research Results](https://mlsanity.com/paper/2010.07474/) (2020)
- [Easy Begun is Half Done: Spatial-Temporal Graph Modeling with ST-Curriculum Dropout](https://mlsanity.com/paper/2211.15182/) (2022)

## Built on by (most cited first, 12 of 12)

- [CCDSReFormer: Traffic Flow Prediction with a Criss-Crossed Dual-Stream Enhanced Rectified Transformer Model](https://mlsanity.com/paper/2403.17753/) (2024)
- [RAM: Replace Attention with MLP for Efficient Multivariate Time Series Forecasting](https://mlsanity.com/paper/2410.24023/) (2024)
- [Spatiotemporal Graph Learning with Direct Volumetric Information Passing and Feature Enhancement](https://mlsanity.com/paper/2409.18013/) (2024)
- [Efficient Traffic Forecasting on Large-Scale Road Network by Regularized Adaptive Graph Convolution](https://mlsanity.com/paper/2506.07179/) (2025)
- [Sim-MSTNet: sim2real based Multi-task SpatioTemporal Network Traffic Forecasting](https://mlsanity.com/paper/2601.21384/) (2026)
- [A Predictive Services Architecture for Efficient Airspace Operations](https://mlsanity.com/paper/2503.17515/) (2025)
- [SPAT: Sensitivity-based Multihead-attention Pruning on Time Series Forecasting Models](https://mlsanity.com/paper/2505.08768/) (2025)
- [FaCTR: Factorized Channel-Temporal Representation Transformers for Efficient Time Series Forecasting](https://mlsanity.com/paper/2506.05597/) (2025)
- [A Knowledge-Guided Cross-Modal Feature Fusion Model for Local Traffic Demand Prediction](https://mlsanity.com/paper/2509.06976/) (2025)
- [Generalising Travel Time Prediction To Varying Route Choices In Urban Networks](https://mlsanity.com/paper/2605.06918/) (2026)
- [More Sensors Only One Field: Rethinking Continual Spatio-Temporal Forecasting](https://mlsanity.com/paper/2609.31325/) (2026)
- [AdaST: Adaptive Coupling for Spatial-Temporal Forecasting](https://mlsanity.com/paper/2609.36119/) (2026)

## Abstract (arXiv)

With the rapid development of the Intelligent Transportation System (ITS), accurate traffic forecasting has emerged as a critical challenge. The key bottleneck lies in capturing the intricate spatio-temporal traffic patterns. In recent years, numerous neural networks with complicated architectures have been proposed to address this issue. However, the advancements in network architectures have encountered diminishing performance gains. In this study, we present a novel component called spatio-temporal adaptive embedding that can yield outstanding results with vanilla transformers. Our proposed Spatio-Temporal Adaptive Embedding transformer (STAEformer) achieves state-of-the-art performance on five real-world traffic forecasting datasets. Further experiments demonstrate that spatio-temporal adaptive embedding plays a crucial role in traffic forecasting by effectively capturing intrinsic spatio-temporal relations and chronological information in traffic time series.