# UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction

arXiv: https://arxiv.org/abs/2402.11838 · HTML: https://mlsanity.com/paper/2402.11838/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 99 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 30 of 43)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction](https://mlsanity.com/paper/1811.05320/) (2018)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [Reasoning with Language Model Prompting: A Survey](https://mlsanity.com/paper/2212.09597/) (2022)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [Sequential Modeling Enables Scalable Learning for Large Vision Models](https://mlsanity.com/paper/2312.00785/) (2023)
- [SimVP: Simpler yet Better Video Prediction](https://mlsanity.com/paper/2206.05099/) (2022)
- [Deep Learning for Spatio-Temporal Data Mining: A Survey](https://mlsanity.com/paper/1906.04928/) (2019)
- [Spatial-Temporal Identity: A Simple yet Effective Baseline for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2208.05233/) (2022)
- [Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2206.09113/) (2022)
- [Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors](https://mlsanity.com/paper/2305.18803/) (2023)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics](https://mlsanity.com/paper/1811.07490/) (2018)
- [Spatio-Temporal Meta-Graph Learning for Traffic Forecasting](https://mlsanity.com/paper/2211.14701/) (2022)
- [Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2206.12126/) (2022)
- [Spatio-Temporal Self-Supervised Learning for Traffic Flow Prediction](https://mlsanity.com/paper/2212.04475/) (2022)
- [Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction](https://mlsanity.com/paper/1901.08518/) (2019)
- [STG2Seq: Spatial-temporal Graph to Sequence Model for Multi-step Passenger Demand Forecasting](https://mlsanity.com/paper/1905.10069/) (2019)
- [Deciphering Spatio-Temporal Graph Forecasting: A Causal Lens and Treatment](https://mlsanity.com/paper/2309.13378/) (2023)
- [Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfer](https://mlsanity.com/paper/2205.13947/) (2022)
- [Domain Adversarial Spatial-Temporal Network: A Transferable Framework for Short-term Traffic Forecasting across Cities](https://mlsanity.com/paper/2202.03630/) (2022)

## Built on by (most cited first, 30 of 87)

- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction](https://mlsanity.com/paper/2408.10269/) (2024)
- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [Empowering Time Series Analysis with Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2405.02358/) (2024)
- [How Can Time Series Analysis Benefit From Multiple Modalities? A Survey and Outlook](https://mlsanity.com/paper/2503.11835/) (2025)
- [UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces](https://mlsanity.com/paper/2411.03859/) (2024)
- [Large Language Models for Mobility Analysis in Transportation Systems: A Survey on Forecasting Tasks](https://mlsanity.com/paper/2405.02357/) (2024)
- [BIGCity: A Universal Spatiotemporal Model for Unified Trajectory and Traffic State Data Analysis](https://mlsanity.com/paper/2412.00953/) (2024)
- [Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learning](https://mlsanity.com/paper/2408.12116/) (2024)
- [A Survey of Generative Techniques for Spatial-Temporal Data Mining](https://mlsanity.com/paper/2405.09592/) (2024)
- [MM-Path: Multi-modal, Multi-granularity Path Representation Learning -- Extended Version](https://mlsanity.com/paper/2411.18428/) (2024)
- [FlexiReg: Flexible Urban Region Representation Learning](https://mlsanity.com/paper/2503.09128/) (2025)
- [A Survey on Multimodal Wearable Sensor-based Human Action Recognition](https://mlsanity.com/paper/2404.15349/) (2024)
- [Prompt-Based Spatio-Temporal Graph Transfer Learning](https://mlsanity.com/paper/2405.12452/) (2024)
- [Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting](https://mlsanity.com/paper/2410.12593/) (2024)
- [Diffusion Transformers as Open-World Spatiotemporal Foundation Models](https://mlsanity.com/paper/2411.12164/) (2024)
- [Large Language-Geometry Model: When LLM meets Equivariance](https://mlsanity.com/paper/2502.11149/) (2025)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [Self-Supervised Representation Learning for Geospatial Objects: A Survey](https://mlsanity.com/paper/2408.12133/) (2024)
- [Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting](https://mlsanity.com/paper/2506.00635/) (2025)
- [Urban Region Pre-training and Prompting: A Graph-based Approach](https://mlsanity.com/paper/2408.05920/) (2024)
- [Noise Matters: Diffusion Model-based Urban Mobility Generation with Collaborative Noise Priors](https://mlsanity.com/paper/2412.05000/) (2024)
- [UniFlow: A Foundation Model for Unified Urban Spatio-Temporal Flow Prediction](https://mlsanity.com/paper/2411.12972/) (2024)
- [A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Prediction](https://mlsanity.com/paper/2408.09815/) (2024)
- [USTBench: Benchmarking and Dissecting Spatiotemporal Reasoning of LLMs as Urban Agents](https://mlsanity.com/paper/2505.17572/) (2025)
- [Domain-Enhanced Dual-Branch Model for Efficient and Interpretable Accident Anticipation](https://mlsanity.com/paper/2507.12755/) (2025)
- [Frequency Enhanced Pre-training for Cross-city Few-shot Traffic Forecasting](https://mlsanity.com/paper/2406.02614/) (2024)
- [UoMo: A Universal Model of Mobile Traffic Forecasting for Wireless Network Optimization](https://mlsanity.com/paper/2410.15322/) (2024)
- [Unveiling the Power of Noise Priors: Enhancing Diffusion Models for Mobile Traffic Prediction](https://mlsanity.com/paper/2501.13794/) (2025)
- [Causal Spatio-Temporal Prediction: An Effective and Efficient Multi-Modal Approach](https://mlsanity.com/paper/2505.17637/) (2025)

## Abstract (arXiv)

Urban spatio-temporal prediction is crucial for informed decision-making, such as traffic management, resource optimization, and emergence response. Despite remarkable breakthroughs in pretrained natural language models that enable one model to handle diverse tasks, a universal solution for spatio-temporal prediction remains challenging Existing prediction approaches are typically tailored for specific spatio-temporal scenarios, requiring task-specific model designs and extensive domain-specific training data. In this study, we introduce UniST, a universal model designed for general urban spatio-temporal prediction across a wide range of scenarios. Inspired by large language models, UniST achieves success through: (i) utilizing diverse spatio-temporal data from different scenarios, (ii) effective pre-training to capture complex spatio-temporal dynamics, (iii) knowledge-guided prompts to enhance generalization capabilities. These designs together unlock the potential of building a universal model for various scenarios Extensive experiments on more than 20 spatio-temporal scenarios demonstrate UniST's efficacy in advancing state-of-the-art performance, especially in few-shot and zero-shot prediction. The datasets and code implementation are released on https://github.com/tsinghua-fib-lab/UniST.