mlsanityData to 14 Oct 2026

Foundation Models for Time Series: A Survey

arXiv 2504.04011 ↗First posted 5 Apr 2025cs.LG cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

Transformer-based foundation models have emerged as a dominant paradigm in time series analysis, offering unprecedented capabilities in tasks such as forecasting, anomaly detection, classification, trend analysis and many more time series analytical tasks. This survey provides a comprehensive overview of the current state of the art pre-trained foundation models, introducing a novel taxonomy to categorize them across several dimensions. Specifically, we classify models by their architecture design, distinguishing between those leveraging patch-based representations and those operating directly on raw sequences. The taxonomy further includes whether the models provide probabilistic or deterministic predictions, and whether they are designed to work with univariate time series or can handle multivariate time series out of the box. Additionally, the taxonomy encompasses model scale and complexity, highlighting differences between lightweight architectures and large-scale foundation models. A unique aspect of this survey is its categorization by the type of objective function employed during training phase. By synthesizing these perspectives, this survey serves as a resource for researchers and practitioners, providing insights into current trends and identifying promising directions for future research in transformer-based time series modeling.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 64 of its 133 references that are in the map. First posted on arXiv 5 Apr 2025.

Builds on

64 of 133 references.

References · 64

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2023A Survey of Large Language Modelscited by 2,576
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  8. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  9. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  10. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  11. 2020GLU Variants Improve Transformercited by 1,804
  12. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
38 more
  1. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  2. 2019Root Mean Square Layer Normalizationcited by 1,480
  3. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  4. 2021Transformers in Vision: A Surveycited by 1,013
  5. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  6. 2024Chronos: Learning the Language of Time Seriescited by 759
  7. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
  8. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  9. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  10. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  11. 2018Deep learning for time series classification: a reviewcited by 576
  12. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  13. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  14. 2020Time Series Forecasting With Deep Learning: A Surveycited by 459
  15. 2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 453
  16. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  17. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
  18. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  19. 2020Query-Key Normalization for Transformerscited by 280
  20. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 240
  21. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  22. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 211
  23. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 208
  24. 2019Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directionscited by 172
  25. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  26. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
  27. 2017Conditional Time Series Forecasting with Convolutional Neural Networkscited by 123
  28. 2024Large Language Models for Time Series: A Surveycited by 114
  29. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  30. 2020Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmarkcited by 108
  31. 2021Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoTcited by 104
  32. 2017Recent Advances in Recurrent Neural Networkscited by 89
  33. 2022Decentralized Training of Foundation Models in Heterogeneous Environmentscited by 87
  34. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 87
  35. 2024Transformers, parallel computation, and logarithmic depthcited by 79
  36. 2019Recurrent Neural Networks (RNNs): A gentle Introduction and Overviewcited by 68
  37. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Modelscited by 65
  38. 2020An Evaluation of Change Point Detection Algorithmscited by 63

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 22.

23 papers.

Sort:

By area:

  1. 2026Reverso: Efficient Time Series Foundation Models for Zero-shot Forecastingcited by 13
  2. 2026Falcon-X: A Time Series Foundation Model for Heterogeneous Multivariate Modelingcited by 10
  3. 2026Explainable Load Forecasting with Covariate-Informed Time Series Foundation Modelscited by 4
  4. 2026Causal Data Augmentation for Robust Fine-Tuning of Tabular Foundation Modelscited by 3
  5. 2026Diversified Scaling Inference in Time Series Foundation Modelscited by 3
  6. 2026Why Do Time Series Models Need Long Context Windows?cited by 3
  7. 2026Into the ORBIT for Time Series: Training Regimes for Foundation Modelscited by 3
  8. 2026XLinear: A Lightweight and Accurate MLP-Based Model for Long-Term Time Series Forecasting with Exogenous Inputscited by 2
  9. 2026TRACE: A Temporal Conditional Estimation for Multimodal Time Series Foundation Modelscited by 2
  10. 2026WPBench: A Comprehensive Benchmark for Wind Power Forecastingcited by 2
  11. 2026VTBench: A Multimodal Framework for Time-Series Classification with Chart-Based Representationscited by 1
  12. 2026Heterogeneous Scientific Foundation Model Collaborationcited by 1
11 more
  1. 2026Towards Resilient and Autonomous Networks: A BlueSky Vision on AI-Native 6Gcited by 1
  2. 2026Ground-Truth Neighborhood Regularization for Reinforcement Learning Post-Training of Time Series Foundation Modelscited by 1
  3. 2025Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamicscited by 0
  4. 2026EngineAD: A Real-World Vehicle Engine Anomaly Detection Datasetcited by 0
  5. 2026Temporal Feature Extractors in EEG Foundation Models: A Controlled Comparison Including a Pretrained Time-Series Modelcited by 0
  6. 2026Exploring Zero-Shot Foundation Models for Multivariate Time Series Anomaly Detectioncited by 0
  7. 2026Learning to Forecast Crop Growth from Earth Observation Datacited by 0
  8. 2026Involving before Evolving: A Vision for Trustworthy Enterprise Digital Twin Engineeringcited by 0
  9. 2026A Unified Scaling Law for Time Series Foundation Modelscited by 0
  10. 2026Inverse Synthesis of Shock Time Series from Shock Response Spectra Using Ensembles of Conditional Variational Autoencoderscited by 0
  11. 2026Beyond Static Positional Encoding: Bayesian Time-aware Attention for Non-Stationary Long-term Time Series Forecastingcited by 0