# Spatial-Temporal Transformer Networks for Traffic Flow Forecasting

arXiv: https://arxiv.org/abs/2001.02908 · HTML: https://mlsanity.com/paper/2001.02908/

## Key facts

- Cited by 180 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2020.

## Built on by (most cited first, 30 of 180)

- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [UniVLA: Learning to Act Anywhere with Task-centric Latent Actions](https://mlsanity.com/paper/2505.06111/) (2025)
- [AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems](https://mlsanity.com/paper/2503.06669/) (2025)
- [Graph Neural Network for Traffic Forecasting: A Survey](https://mlsanity.com/paper/2101.11174/) (2021)
- [Dynamic Graph Convolutional Recurrent Network for Traffic Prediction: Benchmark and Solution](https://mlsanity.com/paper/2104.14917/) (2021)
- [PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction](https://mlsanity.com/paper/2301.07945/) (2023)
- [LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting](https://mlsanity.com/paper/2306.08259/) (2023)
- [Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond](https://mlsanity.com/paper/2405.03520/) (2024)
- [IRASim: A Fine-Grained World Model for Robot Manipulation](https://mlsanity.com/paper/2406.14540/) (2024)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [Spatio-Temporal Meta-Graph Learning for Traffic Forecasting](https://mlsanity.com/paper/2211.14701/) (2022)
- [AirFormer: Predicting Nationwide Air Quality in China with Transformers](https://mlsanity.com/paper/2211.15979/) (2022)
- [IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AI](https://mlsanity.com/paper/2411.00785/) (2024)
- [villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models](https://mlsanity.com/paper/2507.23682/) (2025)
- [Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Version](https://mlsanity.com/paper/2203.15737/) (2022)
- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [AutoCTS: Automated Correlated Time Series Forecasting -- Extended Version](https://mlsanity.com/paper/2112.11174/) (2021)
- [Long-Range Transformers for Dynamic Spatiotemporal Forecasting](https://mlsanity.com/paper/2109.12218/) (2021)
- [UniSkill: Imitating Human Videos via Cross-Embodiment Skill Representations](https://mlsanity.com/paper/2505.08787/) (2025)
- [Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecasting](https://mlsanity.com/paper/2312.00516/) (2023)
- [Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap](https://mlsanity.com/paper/2503.21411/) (2025)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [Embodied AI: From LLMs to World Models](https://mlsanity.com/paper/2509.20021/) (2025)
- [ST-MLP: A Cascaded Spatio-Temporal Linear Framework with Channel-Independence Strategy for Traffic Forecasting](https://mlsanity.com/paper/2308.07496/) (2023)
- [A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulation](https://mlsanity.com/paper/2602.01067/) (2026)
- [LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Prediction](https://mlsanity.com/paper/2304.14343/) (2023)
- [Seeing the forest and the tree: Building representations of both individual and collective dynamics with transformers](https://mlsanity.com/paper/2206.06131/) (2022)
- [LightCTS: A Lightweight Framework for Correlated Time Series Forecasting](https://mlsanity.com/paper/2302.11974/) (2023)
- [TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts](https://mlsanity.com/paper/2403.02600/) (2024)

## Abstract (arXiv)

Traffic forecasting has emerged as a core component of intelligent transportation systems. However, timely accurate traffic forecasting, especially long-term forecasting, still remains an open challenge due to the highly nonlinear and dynamic spatial-temporal dependencies of traffic flows. In this paper, we propose a novel paradigm of Spatial-Temporal Transformer Networks (STTNs) that leverages dynamical directed spatial dependencies and long-range temporal dependencies to improve the accuracy of long-term traffic forecasting. Specifically, we present a new variant of graph neural networks, named spatial transformer, by dynamically modeling directed spatial dependencies with self-attention mechanism to capture realtime traffic conditions as well as the directionality of traffic flows. Furthermore, different spatial dependency patterns can be jointly modeled with multi-heads attention mechanism to consider diverse relationships related to different factors (e.g. similarity, connectivity and covariance). On the other hand, the temporal transformer is utilized to model long-range bidirectional temporal dependencies across multiple time steps. Finally, they are composed as a block to jointly model the spatial-temporal dependencies for accurate traffic prediction. Compared to existing works, the proposed model enables fast and scalable training over a long range spatial-temporal dependencies. Experiment results demonstrate that the proposed model achieves competitive results compared with the state-of-the-arts, especially forecasting long-term traffic flows on real-world PeMS-Bay and PeMSD7(M) datasets.