mlsanityData to 14 Oct 2026

Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook

arXiv 2310.10196 ↗First posted 16 Oct 2023cs.LG cs.AIcited by 110

Not on the map. Why?

Abstract (arXiv)

Temporal data, including time series and spatio-temporal data, are pervasive in real-world applications. Generated in massive volumes by physical and virtual sensors, they record dynamic system behaviors and enable a wide range of downstream tasks. Effectively analyzing such data is crucial to unlocking their rich information content. Recent advances in large language models and other foundation models have accelerated their use in time series and spatio-temporal data mining. These approaches not only improve pattern recognition and reasoning across diverse domains but also support progress toward artificial general intelligence that can understand and process temporal data. In this survey, we present a comprehensive, up-to-date review of large models tailored or adapted for time series and spatio-temporal data along four dimensions: data types, model categories, model scopes, and application areas/tasks. We organize existing work into two main groups: large models for time series analysis (LM4TS) and for spatio-temporal data mining (LM4STD), and further distinguish general-purpose from domain-specific models. We also curate related resources, including datasets, model implementations, and tools, organized by major application areas. Overall, this survey consolidates recent advances and highlights foundations, applications, resources, and open research opportunities in large model-centric temporal data analysis.

Key facts. Cited by 110 papers in the map (data through 14 Oct 2026). Builds on 252 of its 432 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

252 of 432 references.

References · 252

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  10. 2023Segment Anythingcited by 8,234
  11. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  12. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
38 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  10. 2023A Survey of Large Language Modelscited by 2,576
  11. 2022Emergent Abilities of Large Language Modelscited by 2,512
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  13. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  14. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  15. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  16. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  17. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  18. 2021ViViT: A Video Vision Transformercited by 1,620
  19. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  20. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  21. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  22. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  23. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  24. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  25. 2023PaLM 2 Technical Reportcited by 1,162
  26. 2021Video Swin Transformercited by 1,091
  27. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  28. 2021Transformers in Vision: A Surveycited by 1,013
  29. 2023BloombergGPT: A Large Language Model for Financecited by 939
  30. 2023A Survey on Multimodal Large Language Modelscited by 909
  31. 2021Multiscale Vision Transformerscited by 897
  32. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  33. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  34. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
  35. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
  36. 2022A Survey on In-context Learningcited by 721
  37. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  38. 2020Self-supervised Learning: Generative or Contrastivecited by 694

50 most cited of 252 All 252 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 60; 2025: 31; 2026: 14.

110 papers.

Sort:

By area:

  1. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  2. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  3. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  4. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 131
  5. 2024Large Language Models for Time Series: A Surveycited by 114
  6. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 104
  7. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 98
  8. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 95
  9. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 90
  10. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 87
  11. 2024Spatial-Temporal Large Language Model for Traffic Predictioncited by 83
  12. 2025Time-MQA: Time Series Multi-Task Question Answering with Context Enhancementcited by 82
38 more
  1. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 67
  2. 2024Empowering Time Series Analysis with Large Language Models: A Surveycited by 60
  3. 2024Deep Learning for Multivariate Time Series Imputation: A Surveycited by 57
  4. 2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
  5. 2024SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognitioncited by 38
  6. 2025TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agentscited by 37
  7. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  8. 2023Foundation Models for Weather and Climate Data Understanding: A Comprehensive Surveycited by 34
  9. 2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 33
  10. 2024LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling lawcited by 32
  11. 2024Universal Time-Series Representation Learning: A Surveycited by 30
  12. 2024Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detectioncited by 29
  13. 2024Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detectioncited by 26
  14. 2024Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecastingcited by 26
  15. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  16. 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
  17. 2025Foundation Models for Spatio-Temporal Data Science: A Tutorial and Surveycited by 24
  18. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  19. 2024Trajectory Data Management and Mining: A Survey from Deep Learning to the LLM Eracited by 23
  20. 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23
  21. 2025Foundation Models for Time Series: A Surveycited by 23
  22. 2024An overview of domain-specific foundation model: key technologies, applications and challengescited by 19
  23. 2023GNNEvaluator: Evaluating GNN Performance On Unseen Graphs Without Labelscited by 17
  24. 2024Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspectivecited by 17
  25. 2024CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracercited by 17
  26. 2025A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigmscited by 17
  27. 2025Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Modelscited by 15
  28. 2024A Survey of Generative Techniques for Spatial-Temporal Data Miningcited by 14
  29. 2024By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Promptingcited by 12
  30. 2024STBench: Assessing the Ability of Large Language Models in Spatio-Temporal Analysiscited by 11
  31. 2024Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Valuescited by 10
  32. 2024Advancing Time Series Classification with Multimodal Language Modelingcited by 10
  33. 2024Foundation Models for Education: Promises and Prospectscited by 10
  34. 2024Prompt-Based Spatio-Temporal Graph Transfer Learningcited by 10
  35. 2025Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Reviewcited by 10
  36. 2025Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Ratecited by 10
  37. 2026QuitoBench: A High-Quality Open Time Series Forecasting Benchmarkcited by 10
  38. 2024Demonstration of DB-GPT: Next Generation Data Interaction System Empowered by Large Language Modelscited by 9

50 most cited of 110 All 110 (JSON)