# A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting

arXiv: https://arxiv.org/abs/2201.00008 · HTML: https://mlsanity.com/paper/2201.00008/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 61 references that are in the map.
- First posted on arXiv 30 Dec 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [Towards Deeper Graph Neural Networks](https://mlsanity.com/paper/2007.09296/) (2020)
- [Graph Neural Network for Traffic Forecasting: A Survey](https://mlsanity.com/paper/2101.11174/) (2021)
- [Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2012.09641/) (2020)
- [Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2106.12931/) (2021)
- [Spatial-Temporal Transformer Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2001.02908/) (2020)
- [Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction](https://mlsanity.com/paper/1802.08714/) (2018)
- [Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better](https://mlsanity.com/paper/2106.08962/) (2021)
- [Revisiting Spatial-Temporal Similarity: A Deep Learning Framework for Traffic Prediction](https://mlsanity.com/paper/1803.01254/) (2018)
- [Graph Few-shot Learning via Knowledge Transfer](https://mlsanity.com/paper/1910.03053/) (2019)
- [Preserving Dynamic Attention for Long-Term Spatial-Temporal Prediction](https://mlsanity.com/paper/2006.08849/) (2020)
- [Forecaster: A Graph Transformer for Forecasting Spatial and Time-Dependent Data](https://mlsanity.com/paper/1909.04019/) (2019)
- [Interpretable Crowd Flow Prediction with Spatial-Temporal Self-Attention](https://mlsanity.com/paper/2002.09693/) (2020)

## Built on by (most cited first, 9 of 9)

- [Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey](https://mlsanity.com/paper/2303.14483/) (2023)
- [Towards Generative Modeling of Urban Flow through Knowledge-enhanced Denoising Diffusion](https://mlsanity.com/paper/2309.10547/) (2023)
- [A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis](https://mlsanity.com/paper/2310.11959/) (2023)
- [TraffiDent: A Dataset for Understanding the Interplay Between Traffic Dynamics and Incidents](https://mlsanity.com/paper/2407.11477/) (2024)
- [Predicting Large-scale Urban Network Dynamics with Energy-informed Graph Neural Diffusion](https://mlsanity.com/paper/2508.00037/) (2025)
- [EP-GAT: Energy-based Parallel Graph Attention Neural Network for Stock Trend Classification](https://mlsanity.com/paper/2507.08184/) (2025)
- [Capturing Complex Spatial-Temporal Dependencies in Traffic Forecasting: A Self-Attention Approach](https://mlsanity.com/paper/2511.07980/) (2025)
- [Uncertainty-Aware Trajectory Prediction: A Unified Framework Harnessing Positional and Semantic Uncertainties](https://mlsanity.com/paper/2603.29362/) (2026)
- [U-STS-LLM A Unified Spatio-Temporal Steered Large Language Model for Traffic Prediction and Imputation](https://mlsanity.com/paper/2605.11735/) (2026)

## Abstract (arXiv)

We study the forecasting problem for traffic with dynamic, possibly periodical, and joint spatial-temporal dependency between regions. Given the aggregated inflow and outflow traffic of regions in a city from time slots 0 to t-1, we predict the traffic at time t at any region. Prior arts in the area often consider the spatial and temporal dependencies in a decoupled manner or are rather computationally intensive in training with a large number of hyper-parameters to tune. We propose ST-TIS, a novel, lightweight, and accurate Spatial-Temporal Transformer with information fusion and region sampling for traffic forecasting. ST-TIS extends the canonical Transformer with information fusion and region sampling. The information fusion module captures the complex spatial-temporal dependency between regions. The region sampling module is to improve the efficiency and prediction accuracy, cutting the computation complexity for dependency learning from O(n2) to O(n√n), where n is the number of regions. With far fewer parameters than state-of-the-art models, the offline training of our model is significantly faster in terms of tuning and computation (with a reduction of up to 90% on training time and network parameters). Notwithstanding such training efficiency, extensive experiments show that ST-TIS is substantially more accurate in online prediction than state-of-the-art approaches (with an average improvement of up to 9.5% on RMSE, and 12.4% on MAPE).