# STD-PLM: Understanding Both Spatial and Temporal Properties of Spatial-Temporal Data with PLM

arXiv: https://arxiv.org/abs/2407.09096 · HTML: https://mlsanity.com/paper/2407.09096/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 12 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [GMAN: A Graph Multi-Attention Network for Traffic Prediction](https://mlsanity.com/paper/1911.08415/) (2019)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [BRITS: Bidirectional Recurrent Imputation for Time Series](https://mlsanity.com/paper/1805.10572/) (2018)
- [Multi-Time Attention Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/2101.10318/) (2021)
- [OpenAGI: When LLM Meets Domain Experts](https://mlsanity.com/paper/2304.04370/) (2023)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction](https://mlsanity.com/paper/1802.08714/) (2018)
- [PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction](https://mlsanity.com/paper/2301.07945/) (2023)
- [Spatial-Temporal Large Language Model for Traffic Prediction](https://mlsanity.com/paper/2401.10134/) (2024)
- [PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation](https://mlsanity.com/paper/2302.09746/) (2023)
- [STG2Seq: Spatial-temporal Graph to Sequence Model for Multi-step Passenger Demand Forecasting](https://mlsanity.com/paper/1905.10069/) (2019)
- [Low-Rank Autoregressive Tensor Completion for Spatiotemporal Traffic Data Imputation](https://mlsanity.com/paper/2104.14936/) (2021)
- [How Can Large Language Models Understand Spatial-Temporal Data?](https://mlsanity.com/paper/2401.14192/) (2024)

## Built on by (most cited first, 10 of 10)

- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [Urban Computing in the Era of Large Language Models](https://mlsanity.com/paper/2504.02009/) (2025)
- [Breaking the Regional Barrier: Inductive Semantic Topology Learning for Worldwide Air Quality Forecasting](https://mlsanity.com/paper/2601.21899/) (2026)
- [Do We Really Need Adaptive Global Spatial Attention for Traffic Forecasting?](https://mlsanity.com/paper/2607.12462/) (2026)
- [A Survey and Benchmarking of Spatial-Temporal Traffic Data Imputation Models](https://mlsanity.com/paper/2412.04733/) (2024)
- [Reprogramming Vision Foundation Models for Spatio-Temporal Forecasting](https://mlsanity.com/paper/2507.11558/) (2025)
- [FUSE-Traffic: Fusion of Unstructured and Structured Data for Event-aware Traffic Forecasting](https://mlsanity.com/paper/2510.16053/) (2025)
- [Comprehending Spatio-temporal Data via Cinematic Storytelling using Large Language Models](https://mlsanity.com/paper/2510.17301/) (2025)
- [UniGIO: Unified Generative Global In-situ Weather Modeling from Spatiotemporal Incomplete Observations](https://mlsanity.com/paper/2609.22217/) (2026)
- [VaryST: Generalizable Spatio-Temporal Forecasting Beyond the Closed-World Setting](https://mlsanity.com/paper/2610.19813/) (2026)

## Abstract (arXiv)

Spatial-temporal forecasting and imputation are important for real-world intelligent systems. Most existing methods are tailored for individual forecasting or imputation tasks but are not designed for both. Additionally, they are less effective for zero-shot and few-shot learning. While pre-trained language model (PLM) have exhibited strong pattern recognition and reasoning abilities across various tasks, including few-shot and zero-shot learning, their applications in spatial-temporal data understanding has been constrained by insufficient modeling of complex correlations such as the temporal correlations, spatial connectivity, non-pairwise and high-order spatial-temporal correlations within data. In this paper, we propose STD-PLM for understanding both spatial and temporal properties of underlineSpatial-underlineTemporal underlineData with underlinePLM, which is capable of implementing both spatial-temporal forecasting and imputation tasks. STD-PLM understands spatial-temporal correlations via explicitly designed spatial and temporal tokenizers. Topology-aware node embeddings are designed for PLM to comprehend and exploit the topology structure of data in inductive manner. Furthermore, to mitigate the efficiency issues introduced by the PLM, we design a sandglass attention module (SGA) combined with a specific constrained loss function, which significantly improves the model's efficiency while ensuring performance. Extensive experiments demonstrate that STD-PLM exhibits competitive performance and generalization capabilities across the forecasting and imputation tasks on various datasets. Moreover, STD-PLM achieves promising results on both few-shot and zero-shot tasks. The code is made available at hrefhttps://github.com/Hyheng/STD-PLMhttps://github.com/Hyheng/STD-PLM