mlsanityData to 14 Oct 2026

Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey

arXiv 2503.13502 ↗First posted 12 Mar 2025cs.DB cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Spatio-Temporal (ST) data science, which includes sensing, managing, and mining large-scale data across space and time, is fundamental to understanding complex systems in domains such as urban computing, climate science, and intelligent transportation. Traditional deep learning approaches have significantly advanced this field, particularly in the stage of ST data mining. However, these models remain task-specific and often require extensive labeled data. Inspired by the success of Foundation Models (FM), especially large language models, researchers have begun exploring the concept of Spatio-Temporal Foundation Models (STFMs) to enhance adaptability and generalization across diverse ST tasks. Unlike prior architectures, STFMs empower the entire workflow of ST data science, ranging from data sensing, management, to mining, thereby offering a more holistic and scalable approach. Despite rapid progress, a systematic study of STFMs for ST data science remains lacking. This survey aims to provide a comprehensive review of STFMs, categorizing existing methodologies and identifying key research directions to advance ST general intelligence.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 97 of its 174 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

97 of 174 references.

References · 97

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2019Graph WaveNet for Deep Spatial-Temporal Graph Modelingcited by 829
  6. 2024Chronos: Learning the Language of Time Seriescited by 759
  7. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  9. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 566
  10. 2020Adaptive Graph Convolutional Recurrent Network for Traffic Forecastingcited by 530
  11. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  12. 2019GMAN: A Graph Multi-Attention Network for Traffic Predictioncited by 434
38 more
  1. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 398
  2. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  3. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
  4. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  5. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 267
  6. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
  7. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  8. 2023Neural General Circulation Models for Weather and Climatecited by 239
  9. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  10. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 190
  11. 2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 180
  12. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 179
  13. 2023FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Leadcited by 175
  14. 2019Deep Learning for Spatio-Temporal Data Mining: A Surveycited by 172
  15. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 172
  16. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 144
  17. 2023Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Surveycited by 140
  18. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 139
  19. 2022Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecastingcited by 117
  20. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  21. 2024Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generationcited by 100
  22. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 95
  23. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 87
  24. 2024Spatial-Temporal Large Language Model for Traffic Predictioncited by 83
  25. 2023DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Modelscited by 81
  26. 2024UrbanGPT: Spatio-Temporal Large Language Modelscited by 81
  27. 2023UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Webcited by 78
  28. 2022AirFormer: Predicting Nationwide Air Quality in China with Transformerscited by 73
  29. 2023LLMLight: Large Language Models as Traffic Signal Control Agentscited by 73
  30. 2025Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecastingcited by 63
  31. 2024Empowering Time Series Analysis with Large Language Models: A Surveycited by 60
  32. 2022From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answeringcited by 51
  33. 2022Leveraging Language Foundation Models for Human Mobility Forecastingcited by 50
  34. 2024ControlTraj: Controllable Trajectory Generation with Topology-Constrained Diffusion Modelcited by 49
  35. 2023Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoningcited by 45
  36. 2023TrafficGPT: Viewing, Processing and Interacting with Traffic Foundation Modelscited by 45
  37. 2024CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agentscited by 42
  38. 2024Can LLMs Understand Time Series Anomalies?cited by 41

50 most cited of 97 All 97 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 11.

24 papers.

Sort:

By area:

  1. 2025TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Modelscited by 27
  2. 2025GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalizationcited by 19
  3. 2025Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Reviewcited by 10
  4. 2025Large Language Model Powered Intelligent Urban Agents: Concepts, Capabilities, and Applicationscited by 4
  5. 2025Reimagining Urban Science: Scaling Causal Inference with Large Language Modelscited by 3
  6. 2025Physics-Guided Learning of Meteorological Dynamics for Weather Downscaling and Forecastingcited by 3
  7. 2025From Points to Places: Towards Human Mobility-Driven Spatiotemporal Foundation Models via Understanding Placescited by 3
  8. 2025A Modular Multitask Reasoning Framework Integrating Spatio-temporal Models and LLMscited by 3
  9. 2025Urban-R1: Reinforced MLLMs Mitigate Geospatial Biases for Urban General Intelligencecited by 3
  10. 2025Improving Region Representation Learning from Urban Imagery with Noisy Long-Caption Supervisioncited by 2
  11. 2026Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Modelscited by 2
  12. 2026WPBench: A Comprehensive Benchmark for Wind Power Forecastingcited by 2
12 more
  1. 2025PlaceRep: Geospatial Place Representation Learning from Large-Scale Point-of-Interest Datacited by 1
  2. 2025Understanding the Geospatial Reasoning Capabilities of LLMs: A Trajectory Recovery Perspectivecited by 1
  3. 2025FlowNet: Modeling Dynamic Spatio-Temporal Systems via Flow Propagationcited by 1
  4. 2026VISTA: Knowledge-Driven Vessel Trajectory Imputation with Repair Provenancecited by 1
  5. 2026UrbanFM: Scaling Urban Spatio-Temporal Foundation Modelscited by 1
  6. 2026Learning from Yesterday's Error: An Efficient Online Learning Method for Traffic Demand Predictioncited by 1
  7. 2026WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecastingcited by 1
  8. 2026Context-Enriched Natural Language Descriptions of Vessel Trajectoriescited by 0
  9. 2026Learning Spatio-Temporal Foundation Models from Pure Synthetic Datacited by 0
  10. 2026STHMoE: Hypergraph-Enhanced Heterogeneous Dependency Coordination for LLM-Based Urban Traffic Data Forecastingcited by 0
  11. 2026Industrial Kinematic Trajectory Model (IKTM): Coordinate-Free Autoregressive Generatorcited by 0
  12. 2026LLM-enhanced spatio-temporal learning for grid-level docked bike sharing demand predictioncited by 0