# OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction

arXiv: https://arxiv.org/abs/2408.10269 · HTML: https://mlsanity.com/paper/2408.10269/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 57 references that are in the map.
- First posted on arXiv 16 Aug 2024.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks](https://mlsanity.com/paper/2005.11650/) (2020)
- [Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting](https://mlsanity.com/paper/2007.02842/) (2020)
- [GMAN: A Graph Multi-Attention Network for Traffic Prediction](https://mlsanity.com/paper/1911.08415/) (2019)
- [T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction](https://mlsanity.com/paper/1811.05320/) (2018)
- [Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting](https://mlsanity.com/paper/2106.12931/) (2021)
- [Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey](https://mlsanity.com/paper/2303.14483/) (2023)
- [PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction](https://mlsanity.com/paper/2301.07945/) (2023)
- [FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective](https://mlsanity.com/paper/2311.06190/) (2023)
- [LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting](https://mlsanity.com/paper/2306.08259/) (2023)
- [Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2206.09113/) (2022)
- [Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting](https://mlsanity.com/paper/2302.14829/) (2023)
- [UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2402.11838/) (2024)
- [Spatial-Temporal Large Language Model for Traffic Prediction](https://mlsanity.com/paper/2401.10134/) (2024)
- [UrbanGPT: Spatio-Temporal Large Language Models](https://mlsanity.com/paper/2403.00813/) (2024)
- [Spatio-Temporal Self-Supervised Learning for Traffic Flow Prediction](https://mlsanity.com/paper/2212.04475/) (2022)
- [Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs](https://mlsanity.com/paper/2202.08408/) (2022)
- [Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction](https://mlsanity.com/paper/1901.08518/) (2019)
- [Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfer](https://mlsanity.com/paper/2205.13947/) (2022)

## Built on by (most cited first, 27 of 27)

- [Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey](https://mlsanity.com/paper/2503.13502/) (2025)
- [UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces](https://mlsanity.com/paper/2411.03859/) (2024)
- [BIGCity: A Universal Spatiotemporal Model for Unified Trajectory and Traffic State Data Analysis](https://mlsanity.com/paper/2412.00953/) (2024)
- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [GATSim: Urban Mobility Simulation with Generative Agents](https://mlsanity.com/paper/2506.23306/) (2025)
- [Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting](https://mlsanity.com/paper/2506.00635/) (2025)
- [Learning Transferable Sensor Models via Language-Informed Pretraining](https://mlsanity.com/paper/2603.11950/) (2026)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [RePST: Language Model Empowered Spatio-Temporal Forecasting via Semantic-Oriented Reprogramming](https://mlsanity.com/paper/2408.14505/) (2024)
- [Urban Computing in the Era of Large Language Models](https://mlsanity.com/paper/2504.02009/) (2025)
- [Damba-ST: Domain-Adaptive Mamba for Efficient Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2506.18939/) (2025)
- [Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Models](https://mlsanity.com/paper/2601.12083/) (2026)
- [Enhancing Ride-Hailing Forecasting at DiDi with Multi-View Geospatial Representation Learning from the Web](https://mlsanity.com/paper/2602.10502/) (2026)
- [GraFSTNet: Graph-based Frequency SpatioTemporal Network for Cellular Traffic Prediction](https://mlsanity.com/paper/2602.13282/) (2026)
- [WPBench: A Comprehensive Benchmark for Wind Power Forecasting](https://mlsanity.com/paper/2609.24444/) (2026)
- [Breaking Data Silos: Towards Open and Scalable Mobility Foundation Models via Generative Continual Learning](https://mlsanity.com/paper/2506.06694/) (2025)
- [Generalising Traffic Forecasting to Regions without Traffic Observations](https://mlsanity.com/paper/2508.08947/) (2025)
- [UrbanFM: Scaling Urban Spatio-Temporal Foundation Models](https://mlsanity.com/paper/2602.20677/) (2026)
- [Learning from Yesterday's Error: An Efficient Online Learning Method for Traffic Demand Prediction](https://mlsanity.com/paper/2602.21757/) (2026)
- [Incorporating Long-term Data in Training Short-term Traffic Prediction Model](https://mlsanity.com/paper/2410.14726/) (2024)
- [Learning from Complexity: Exploring Dynamic Sample Pruning of Spatio-Temporal Training](https://mlsanity.com/paper/2602.19113/) (2026)
- [GeoGNN: Time Series Geo-Localization using Two-Tower Graph Neural Networks](https://mlsanity.com/paper/2606.08303/) (2026)
- [Multimodal Spatiotemporal-Frequency Fusion with Peak Enhancement for Cellular Traffic Forecasting](https://mlsanity.com/paper/2607.07016/) (2026)
- [Learning Spatio-Temporal Foundation Models from Pure Synthetic Data](https://mlsanity.com/paper/2607.16251/) (2026)
- [Neural Architecture Search for Traffic Prediction: A Survey of Methods, Challenges, and Future Directions](https://mlsanity.com/paper/2607.26467/) (2026)
- [A Multi-Resolution Multi-Domain Pre-Training Framework for Universal Traffic Forecasting](https://mlsanity.com/paper/2609.13878/) (2026)
- [STHMoE: Hypergraph-Enhanced Heterogeneous Dependency Coordination for LLM-Based Urban Traffic Data Forecasting](https://mlsanity.com/paper/2609.15172/) (2026)

## Abstract (arXiv)

Accurate traffic forecasting is crucial for effective urban planning and transportation management, enabling efficient resource allocation and enhanced travel experiences. However, existing models often face limitations in generalization, struggling with zero-shot prediction on unseen regions and cities, as well as diminished long-term accuracy. This is primarily due to the inherent challenges in handling the spatial and temporal heterogeneity of traffic data, coupled with the significant distribution shift across time and space. In this work, we aim to unlock new possibilities for building versatile, resilient and adaptive spatio-temporal foundation models for traffic prediction. To achieve this goal, we introduce a novel foundation model, named OpenCity, that can effectively capture and normalize the underlying spatio-temporal patterns from diverse data characteristics, facilitating zero-shot generalization across diverse urban environments. OpenCity integrates the Transformer architecture with graph neural networks to model the complex spatio-temporal dependencies in traffic data. By pre-training OpenCity on large-scale, heterogeneous traffic datasets, we enable the model to learn rich, generalizable representations that can be seamlessly applied to a wide range of traffic forecasting scenarios. Experimental results demonstrate that OpenCity exhibits exceptional zero-shot predictive performance. Moreover, OpenCity showcases promising scaling laws, suggesting the potential for developing a truly one-for-all traffic prediction solution that can adapt to new urban contexts with minimal overhead. We made our proposed OpenCity model open-source and it is available at the following link: https://github.com/HKUDS/OpenCity.