mlsanityData to 7 Oct 2026

Deep Time Series Models: A Comprehensive Survey and Benchmark

arXiv 2407.13278 ↗First posted 18 Jul 2024cs.LGcited by 191

Not on the map. Why?

Abstract (arXiv)

Time series, characterized by a sequence of data points organized in a discrete-time order, are ubiquitous in real-world scenarios. Unlike other data modalities, time series present unique challenges in learning and modeling due to their intricate and dynamic nature, including the entanglement of nonlinear patterns and time-variant trends. Recent years have witnessed remarkable breakthroughs in time series analysis, with techniques shifting from traditional statistical methods to contemporary deep learning models. In this paper, we delve into the design of deep time series models across various analysis tasks and review the existing literature from two perspectives: basic modules and model architectures. Further, we develop and release Time Series Library (TSLib) as a fair benchmark of deep time series models for diverse analysis tasks. TSLib implements 41 prominent models, including both small- and large-scale time series models, covers 30 datasets from different domains, and supports 5 prevalent analysis tasks. Based on TSLib, we evaluate 16 popular deep time series models and 6 advanced time series foundation models. Empirical findings indicate that models with specific structures are apt only at distinct analytical tasks, providing insights for research and adoption of deep time series models. Code and datasets are available at https://github.com/thuml/Time-Series-Library.

Key facts. Cited by 191 papers in the map (data through 7 Oct 2026). Builds on 143 of its 241 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

143 of 241 references.

References · 143

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  11. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  12. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
38 more
  1. 2023Segment Anythingcited by 8,158
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2014Going Deeper with Convolutionscited by 4,231
  12. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  13. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  14. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  15. 2018Neural Ordinary Differential Equationscited by 3,733
  16. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  17. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  18. 2023A Survey of Large Language Modelscited by 2,537
  19. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  20. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  21. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  22. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  23. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  24. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  25. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  26. 2022Visual Prompt Tuningcited by 1,674
  27. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  28. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  29. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  30. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  31. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  32. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  33. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  34. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  35. 2017Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecastingcited by 1,125
  36. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  37. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  38. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906

50 most cited of 143 All 143 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 94; 2026: 84.

191 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey of Deep Learning for Multivariate Time Series Forecasting: A Channel Strategy Perspectivecited by 58
  2. 2025Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrativecited by 35
  3. 2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 30
  4. 2025EEG Foundation Models: A Critical Review of Current Progress and Future Directionscited by 30
  5. 2025Temporal Query Network for Efficient Multivariate Time Series Forecastingcited by 29
  6. 2025Kronos: A Foundation Model for the Language of Financial Marketscited by 25
  7. 2026Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scalingcited by 24
  8. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 23
  9. 2025FinTSB: A Comprehensive and Practical Benchmark for Financial Time Series Forecastingcited by 23
  10. 2025Time-o1: Time-Series Forecasting Needs Transformed Label Alignmentcited by 22
  11. 2024xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memoriescited by 18
  12. 2025Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Seriescited by 16
38 more
  1. 2024Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learningcited by 15
  2. 2025Improving Time Series Forecasting via Instance-aware Post-hoc Revisioncited by 14
  3. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Modelscited by 14
  4. 2025DistDF: Time-Series Forecasting Needs Joint-Distribution Wasserstein Alignmentcited by 13
  5. 2025TimeDistill: Efficient Long-Term Time Series Forecasting with MLP via Cross-Architecture Distillationcited by 12
  6. 2025Breaking Silos: Adaptive Model Fusion Unlocks Better Time Series Forecastingcited by 12
  7. 2025Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insightscited by 11
  8. 2025AlphaCast: A Human Wisdom-LLM Intelligence Co-Reasoning Framework for Interactive Time Series Forecastingcited by 11
  9. 2025TimesBERT: A BERT-Style Foundation Model for Time Series Understandingcited by 10
  10. 2025Time Series Analysis in Frequency Domain: A Survey of Open Challenges, Opportunities and Benchmarkscited by 10
  11. 2025MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Seriescited by 9
  12. 2025There are no Champions in Supervised Long-Term Time Series Forecastingcited by 8
  13. 2025Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Modelscited by 8
  14. 2025CATS: Mitigating Correlation Shift for Multivariate Time Series Classificationcited by 7
  15. 2025AD-AGENT: A Multi-agent Framework for End-to-end Anomaly Detectioncited by 7
  16. 2025Teaching Time Series to See and Speak: Forecasting with Aligned Visual and Textual Perspectivescited by 7
  17. 2025Time-RA: Towards Time Series Reasoning for Anomaly Diagnosis with LLM Feedbackcited by 7
  18. 2025TimeRecipe: A Time-Series Forecasting Recipe via Benchmarking Module Level Effectivenesscited by 6
  19. 2024Metadata Matters for Time Series: Informative Forecasting with Transformerscited by 5
  20. 2025ReFocus: Reinforcing Mid-Frequency and Key-Frequency Modeling for Multivariate Time Series Forecastingcited by 5
  21. 2025CITRAS: Covariate-Informed Transformer for Time Series Forecastingcited by 5
  22. 2025A Comparative Study of Transformer-Based Models for Multi-Horizon Blood Glucose Predictioncited by 5
  23. 2025LETS Forecast: Learning Embedology for Time Series Forecastingcited by 5
  24. 2025TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrievalcited by 5
  25. 2025U-Cast: Learning Hierarchical Structures for High-Dimensional Time Series Forecastingcited by 5
  26. 2025Fidel-TS: A High-Fidelity Multimodal Benchmark for Time Series Forecastingcited by 5
  27. 2026Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Seriescited by 5
  28. 2025Using Causality for Enhanced Prediction of Web Traffic Time Seriescited by 4
  29. 2025Exploring Accuracy Law for Deep Time Series Forecasters: An Empirical Studycited by 4
  30. 2024Toward Physics-guided Time Series Embeddingcited by 3
  31. 2024LiNo: Advancing Recursive Residual Decomposition of Linear and Nonlinear Patterns for Robust Time Series Forecastingcited by 3
  32. 2025Pre-training Epidemic Time Series Forecasters with Compartmental Prototypescited by 3
  33. 2025Augmented Invertible Koopman Autoencoder for long-term time series forecastingcited by 3
  34. 2025MSDformer: Multi-scale Discrete Transformer For Time Series Generationcited by 3
  35. 2025MoTime: A Dataset Suite for Multimodal Time Series Forecastingcited by 3
  36. 2025Augur: Modeling Covariate Causal Associations in Time Series via Large Language Modelscited by 3
  37. 2025SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Seriescited by 3
  38. 2026MambaSL: Exploring Single-Layer Mamba for Time Series Classificationcited by 3

50 most cited of 191 All 191 (JSON)