# Efficient Large-Scale Traffic Forecasting with Transformers: A Spatial Data Management Perspective

arXiv: https://arxiv.org/abs/2412.09972 · HTML: https://mlsanity.com/paper/2412.09972/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 62 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 18 of 18)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [GMAN: A Graph Multi-Attention Network for Traffic Prediction](https://mlsanity.com/paper/1911.08415/) (2019)
- [Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2106.12931/) (2021)
- [Dynamic Graph Convolutional Recurrent Network for Traffic Prediction: Benchmark and Solution](https://mlsanity.com/paper/2104.14917/) (2021)
- [Graph Neural Controlled Differential Equations for Traffic Forecasting](https://mlsanity.com/paper/2112.03558/) (2021)
- [LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting](https://mlsanity.com/paper/2306.08259/) (2023)
- [A Generalization of ViT/MLP-Mixer to Graphs](https://mlsanity.com/paper/2212.13350/) (2022)
- [DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models](https://mlsanity.com/paper/2301.13629/) (2023)
- [RAGraph: A General Retrieval-Augmented Graph Learning Framework](https://mlsanity.com/paper/2410.23855/) (2024)
- [Uncertainty Quantification via Spatial-Temporal Tweedie Model for Zero-inflated and Long-tail Travel Demand Prediction](https://mlsanity.com/paper/2306.09882/) (2023)
- [RPMixer: Shaking Up Time Series Forecasting with Random Projections for Large Spatial-Temporal Data](https://mlsanity.com/paper/2402.10487/) (2024)
- [Rethinking Sensors Modeling: Hierarchical Information Enhanced Traffic Forecasting](https://mlsanity.com/paper/2309.11284/) (2023)
- [PatchGT: Transformer over Non-trainable Clusters for Learning Graph Representations](https://mlsanity.com/paper/2211.14425/) (2022)
- [MILLION: A General Multi-Objective Framework with Controllable Risk for Portfolio Management](https://mlsanity.com/paper/2412.03038/) (2024)
- [COMET: NFT Price Prediction with Wallet Profiling](https://mlsanity.com/paper/2405.10640/) (2024)

## Built on by (most cited first, 26 of 26)

- [Efficient Multivariate Time Series Forecasting via Calibrated Language Models with Privileged Knowledge Distillation](https://mlsanity.com/paper/2505.02138/) (2025)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting](https://mlsanity.com/paper/2506.00635/) (2025)
- [How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learning](https://mlsanity.com/paper/2510.04908/) (2025)
- [A Survey of Physics-Informed AI for Complex Urban Systems](https://mlsanity.com/paper/2506.13777/) (2025)
- [S2Transformer: Scalable Structured Transformers for Global Station Weather Forecasting](https://mlsanity.com/paper/2509.19648/) (2025)
- [FaST: Efficient and Effective Long-Horizon Forecasting for Large-Scale Spatial-Temporal Graphs via Mixture-of-Experts](https://mlsanity.com/paper/2601.05174/) (2026)
- [Position: The Inevitable End of One-Architecture-Fits-All-Domains in Time Series Forecasting](https://mlsanity.com/paper/2602.01736/) (2026)
- [Effective Dataset Distillation for Spatio-Temporal Forecasting with Bi-dimensional Compression](https://mlsanity.com/paper/2603.10410/) (2026)
- [Rigid-Deformation Decomposition AI Framework for 3D Spatio-Temporal Prediction of Vehicle Collision Dynamics](https://mlsanity.com/paper/2503.19712/) (2025)
- [Efficient Traffic Forecasting on Large-Scale Road Network by Regularized Adaptive Graph Convolution](https://mlsanity.com/paper/2506.07179/) (2025)
- [HieraMix: A Hierarchical MLP-Mixer for Large-Scale Traffic Forecasting](https://mlsanity.com/paper/2512.07854/) (2025)
- [Nested Spatio-Temporal Time Series Forecasting](https://mlsanity.com/paper/2605.16447/) (2026)
- [Do We Really Need Adaptive Global Spatial Attention for Traffic Forecasting?](https://mlsanity.com/paper/2607.12462/) (2026)
- [Robust Spatiotemporal Forecasting Using Adaptive Deep-Unfolded Variational Mode Decomposition](https://mlsanity.com/paper/2509.00703/) (2025)
- [DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2509.14868/) (2025)
- [CometNet: Contextual Motif-guided Long-term Time Series Forecasting](https://mlsanity.com/paper/2511.08049/) (2025)
- [Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training](https://mlsanity.com/paper/2602.19113/) (2026)
- [Continuous Cross-Domain Traffic State Prediction via Memory-Augmented Graph Liquid Time-Constant Networks](https://mlsanity.com/paper/2606.15807/) (2026)
- [Structure-Aware Graph Multi-Task Learning for Dynamic Sparse OD Demand Prediction](https://mlsanity.com/paper/2606.21022/) (2026)
- [Frequency-Domain Multi-Modality Transportation Modeling](https://mlsanity.com/paper/2607.08475/) (2026)
- [Fourier Geometric Wind Power Forecasting with Numerical Weather Prediction](https://mlsanity.com/paper/2607.17095/) (2026)
- [Neural Architecture Search for Traffic Prediction: A Survey of Methods, Challenges, and Future Directions](https://mlsanity.com/paper/2607.26467/) (2026)
- [Learning Local Heterogeneity and Cross-Region Context for Large-Scale Traffic Forecasting](https://mlsanity.com/paper/2609.27637/) (2026)
- [FreSia: Frequency-Semantic Instantiation and Alignment for Multivariate Time Series Analysis](https://mlsanity.com/paper/2610.05726/) (2026)
- [VaryST: Generalizable Spatio-Temporal Forecasting Beyond the Closed-World Setting](https://mlsanity.com/paper/2610.19813/) (2026)

## Abstract (arXiv)

Road traffic forecasting is crucial in real-world intelligent transportation scenarios like traffic dispatching and path planning in city management and personal traveling. Spatio-temporal graph neural networks (STGNNs) stand out as the mainstream solution in this task. Nevertheless, the quadratic complexity of remarkable dynamic spatial modeling-based STGNNs has become the bottleneck over large-scale traffic data. From the spatial data management perspective, we present a novel Transformer framework called PatchSTG to efficiently and dynamically model spatial dependencies for large-scale traffic forecasting with interpretability and fidelity. Specifically, we design a novel irregular spatial patching to reduce the number of points involved in the dynamic calculation of Transformer. The irregular spatial patching first utilizes the leaf K-dimensional tree (KDTree) to recursively partition irregularly distributed traffic points into leaf nodes with a small capacity, and then merges leaf nodes belonging to the same subtree into occupancy-equaled and non-overlapped patches through padding and backtracking. Based on the patched data, depth and breadth attention are used interchangeably in the encoder to dynamically learn local and global spatial knowledge from points in a patch and points with the same index of patches. Experimental results on four real world large-scale traffic datasets show that our PatchSTG achieves train speed and memory utilization improvements up to 10× and 4× with the state-of-the-art performance.