# PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction

arXiv: https://arxiv.org/abs/2301.07945 · HTML: https://mlsanity.com/paper/2301.07945/

## Key facts

- Cited by 137 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 19 Jan 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [GMAN: A Graph Multi-Attention Network for Traffic Prediction](https://mlsanity.com/paper/1911.08415/) (2019)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2012.09641/) (2020)
- [Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2106.12931/) (2021)
- [Spatial-Temporal Transformer Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2001.02908/) (2020)
- [Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction](https://mlsanity.com/paper/1802.08714/) (2018)
- [Graph Neural Controlled Differential Equations for Traffic Forecasting](https://mlsanity.com/paper/2112.03558/) (2021)
- [Multi-Range Attentive Bicomponent Graph Convolutional Network for Traffic Forecasting](https://mlsanity.com/paper/1911.12093/) (2019)
- [Self-supervised Trajectory Representation Learning with Temporal Regularities and Travel Semantics](https://mlsanity.com/paper/2211.09510/) (2022)
- [Coupled Layer-wise Graph Convolution for Transportation Demand Prediction](https://mlsanity.com/paper/2012.08080/) (2020)
- [Traffic Flow Forecasting with Spatial-Temporal Graph Diffusion Network](https://mlsanity.com/paper/2110.04038/) (2021)
- [STDEN: Towards Physics-Guided Neural Networks for Traffic Flow Prediction](https://mlsanity.com/paper/2209.00225/) (2022)
- [Preserving Dynamic Attention for Long-Term Spatial-Temporal Prediction](https://mlsanity.com/paper/2006.08849/) (2020)
- [FC-GAGA: Fully Connected Gated Graph Architecture for Spatio-Temporal Traffic Forecasting](https://mlsanity.com/paper/2007.15531/) (2020)
- [Learning dynamic and hierarchical traffic spatiotemporal features with Transformer](https://mlsanity.com/paper/2104.05163/) (2021)

## Built on by (most cited first, 30 of 137)

- [Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis](https://mlsanity.com/paper/2310.06119/) (2023)
- [Spatial-Temporal Large Language Model for Traffic Prediction](https://mlsanity.com/paper/2401.10134/) (2024)
- [A Survey on Diffusion Models for Time Series and Spatio-Temporal Data](https://mlsanity.com/paper/2404.18886/) (2024)
- [How Can Large Language Models Understand Spatial-Temporal Data?](https://mlsanity.com/paper/2401.14192/) (2024)
- [Mixture-of-Linear-Experts for Long-term Time Series Forecasting](https://mlsanity.com/paper/2312.06786/) (2023)
- [Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecasting](https://mlsanity.com/paper/2312.00516/) (2023)
- [OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction](https://mlsanity.com/paper/2408.10269/) (2024)
- [STG-Mamba: Spatial-Temporal Graph Learning via Selective State Space Model](https://mlsanity.com/paper/2403.12418/) (2024)
- [Irregular Traffic Time Series Forecasting Based on Asynchronous Spatio-Temporal Graph Convolutional Network](https://mlsanity.com/paper/2308.16818/) (2023)
- [Towards Explainable Traffic Flow Prediction with Large Language Models](https://mlsanity.com/paper/2404.02937/) (2024)
- [FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Prediction](https://mlsanity.com/paper/2405.17898/) (2024)
- [BIGCity: A Universal Spatiotemporal Model for Unified Trajectory and Traffic State Data Analysis](https://mlsanity.com/paper/2412.00953/) (2024)
- [A Survey of Generative Techniques for Spatial-Temporal Data Mining](https://mlsanity.com/paper/2405.09592/) (2024)
- [STAEformer: Spatio-Temporal Adaptive Embedding Makes Vanilla Transformer SOTA for Traffic Forecasting](https://mlsanity.com/paper/2308.10425/) (2023)
- [Online Test-Time Adaptation of Spatial-Temporal Traffic Flow Forecasting](https://mlsanity.com/paper/2401.04148/) (2024)
- [STD-PLM: Understanding Both Spatial and Temporal Properties of Spatial-Temporal Data with PLM](https://mlsanity.com/paper/2407.09096/) (2024)
- [Diffusion Transformers as Open-World Spatiotemporal Foundation Models](https://mlsanity.com/paper/2411.12164/) (2024)
- [STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting](https://mlsanity.com/paper/2410.00385/) (2024)
- [A Systematic Literature Review of Spatio-Temporal Graph Neural Network Models for Time Series Forecasting and Classification](https://mlsanity.com/paper/2410.22377/) (2024)
- [Full Bayesian Significance Testing for Neural Networks](https://mlsanity.com/paper/2401.13335/) (2024)
- [ChatTraffic: Text-to-Traffic Generation via Diffusion Model](https://mlsanity.com/paper/2311.16203/) (2023)
- [SpoT-Mamba: Learning Long-Range Dependency on Spatio-Temporal Graphs with Selective State Spaces](https://mlsanity.com/paper/2406.11244/) (2024)
- [UniFlow: A Foundation Model for Unified Urban Spatio-Temporal Flow Prediction](https://mlsanity.com/paper/2411.12972/) (2024)
- [XXLTraffic: Expanding and Extremely Long Traffic forecasting beyond test adaptation](https://mlsanity.com/paper/2406.12693/) (2024)
- [CCDSReFormer: Traffic Flow Prediction with a Criss-Crossed Dual-Stream Enhanced Rectified Transformer Model](https://mlsanity.com/paper/2403.17753/) (2024)
- [Dynamic Trend Fusion Module for Traffic Flow Prediction](https://mlsanity.com/paper/2501.10796/) (2025)
- [Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmap](https://mlsanity.com/paper/2512.04680/) (2025)
- [TimelyGPT: Extrapolatable Transformer Pre-training for Long-term Time-Series Forecasting in Healthcare](https://mlsanity.com/paper/2312.00817/) (2023)
- [STMGF: An Effective Spatial-Temporal Multi-Granularity Framework for Traffic Forecasting](https://mlsanity.com/paper/2404.05774/) (2024)
- [UniTE: A Survey and Unified Pipeline for Pre-training Spatiotemporal Trajectory Embeddings](https://mlsanity.com/paper/2407.12550/) (2024)

## Abstract (arXiv)

As a core technology of Intelligent Transportation System, traffic flow prediction has a wide range of applications. The fundamental challenge in traffic flow prediction is to effectively model the complex spatial-temporal dependencies in traffic data. Spatial-temporal Graph Neural Network (GNN) models have emerged as one of the most promising methods to solve this problem. However, GNN-based models have three major limitations for traffic prediction: i) Most methods model spatial dependencies in a static manner, which limits the ability to learn dynamic urban traffic patterns; ii) Most methods only consider short-range spatial information and are unable to capture long-range spatial dependencies; iii) These methods ignore the fact that the propagation of traffic conditions between locations has a time delay in traffic systems. To this end, we propose a novel Propagation Delay-aware dynamic long-range transFormer, namely PDFormer, for accurate traffic flow prediction. Specifically, we design a spatial self-attention module to capture the dynamic spatial dependencies. Then, two graph masking matrices are introduced to highlight spatial dependencies from short- and long-range views. Moreover, a traffic delay-aware feature transformation module is proposed to empower PDFormer with the capability of explicitly modeling the time delay of spatial information propagation. Extensive experimental results on six real-world public traffic datasets show that our method can not only achieve state-of-the-art performance but also exhibit competitive computational efficiency. Moreover, we visualize the learned spatial-temporal attention map to make our model highly interpretable.