mlsanityData to 7 Oct 2026

Are Language Models Actually Useful for Time Series Forecasting?

arXiv 2406.16964 ↗First posted 22 Jun 2024cs.LG cs.AIcited by 163

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are being applied to time series forecasting. But are language models actually useful for time series? In a series of ablation studies on three recent and popular LLM-based time series forecasting methods, we find that removing the LLM component or replacing it with a basic attention layer does not degrade forecasting performance -- in most cases, the results even improve! We also find that despite their significant computational cost, pretrained LLMs do no better than models trained from scratch, do not represent the sequential dependencies in time series, and do not assist in few-shot settings. Additionally, we explore time series encoders and find that patching and attention structures perform similarly to LLM-based forecasters.

Key facts. Cited by 163 papers in the map (data through 7 Oct 2026). Builds on 32 of its 51 references that are in the map. First posted on arXiv 22 Jun 2024.

Builds on

32 of 51 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  4. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  5. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  6. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  7. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  8. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  9. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  10. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  11. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  12. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
20 more
  1. 2024Chronos: Learning the Language of Time Seriescited by 712
  2. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  3. 2023A decoder-only foundation model for time-series forecastingcited by 650
  4. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  5. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  6. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  7. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  8. 2019Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019cited by 270
  9. 2021Monash Time Series Forecasting Archivecited by 256
  10. 2023FITS: Modeling Time Series with 10k Parameterscited by 185
  11. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 170
  12. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  13. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  14. 2023Large Language Models are Few-Shot Health Learnerscited by 115
  15. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
  16. 2024Language Models Still Struggle to Zero-shot Reason about Time Seriescited by 81
  17. 2021Time Series Prediction using Deep Learning Methods in Healthcarecited by 71
  18. 2023Learning to Embed Time Series Patches Independentlycited by 34
  19. 2024Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicatorscited by 32
  20. 2023Leveraging Vision-Language Models for Granular Market Change Predictioncited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 85; 2026: 58.

163 papers.

Sort:

By area:

  1. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  2. 2025MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answeringcited by 50
  3. 2024Can LLMs Understand Time Series Anomalies?cited by 41
  4. 2025TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecastercited by 36
  5. 2024SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognitioncited by 35
  6. 2025Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrativecited by 35
  7. 2025Harnessing Vision Models for Time Series Analysis: A Surveycited by 26
  8. 2024Zero-shot forecasting of chaotic systemscited by 25
  9. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 23
  10. 2025SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Modelscited by 22
  11. 2025FoNE: Precise Single-Token Number Embeddings via Fourier Featurescited by 21
  12. 2025A Survey of Reasoning and Agentic Systems in Time Series with Large Language Modelscited by 21
38 more
  1. 2025True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statisticscited by 20
  2. 2025BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamicscited by 16
  3. 2025CT-PatchTST: Channel-Time Patch Time-Series Transformer for Long-Term Renewable Energy Forecastingcited by 15
  4. 2025Out-of-Distribution Generalization in Time Series: A Surveycited by 15
  5. 2025Decision-informed Neural Networks with Large Language Model Integration for Portfolio Optimizationcited by 14
  6. 2025SciTS: Scientific Time Series Understanding and Generation with LLMscited by 13
  7. 2025Breaking Silos: Adaptive Model Fusion Unlocks Better Time Series Forecastingcited by 12
  8. 2024ViTime: Foundation Model for Time Series Forecasting Powered by Vision Intelligencecited by 11
  9. 2025BEDTime: A Unified Benchmark for Automatically Describing Time Seriescited by 11
  10. 2026Reverso: Efficient Time Series Foundation Models for Zero-shot Forecastingcited by 11
  11. 2024A Systematic Literature Review of Spatio-Temporal Graph Neural Network Models for Time Series Forecasting and Classificationcited by 9
  12. 2024Financial Fine-tuning a Large Time Series Modelcited by 8
  13. 2025A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptionscited by 8
  14. 2025There are no Champions in Supervised Long-Term Time Series Forecastingcited by 8
  15. 2025Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecastingcited by 8
  16. 2024Understanding Why Large Language Models Can Be Ineffective in Time Series Analysis: The Impact of Modality Alignmentcited by 7
  17. 2025TempoGPT: Enhancing Time Series Reasoning via Quantizing Embeddingcited by 7
  18. 2025From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?cited by 7
  19. 2025Teaching Time Series to See and Speak: Forecasting with Aligned Visual and Textual Perspectivescited by 7
  20. 2025Revisiting Multivariate Time Series Forecasting with Missing Valuescited by 7
  21. 2024Repurposing Foundation Model for Generalizable Medical Time Series Classificationcited by 6
  22. 2024Adversarial Vulnerabilities in Large Language Models for Time Series Forecastingcited by 6
  23. 2025Using Pre-trained LLMs for Multivariate Time Series Forecastingcited by 6
  24. 2025TimeRecipe: A Time-Series Forecasting Recipe via Benchmarking Module Level Effectivenesscited by 6
  25. 2025Evaluating LLMs on Real-World Forecasting Against Expert Forecasterscited by 6
  26. 2025CC-Time: Cross-Model and Cross-Modality Time Series Forecastingcited by 6
  27. 2025Text Reinforcement for Multimodal Time Series Forecastingcited by 6
  28. 2025ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Modelscited by 6
  29. 2026From Observations to States: Latent Time Series Forecastingcited by 6
  30. 2024Metadata Matters for Time Series: Informative Forecasting with Transformerscited by 5
  31. 2025U-Cast: Learning Hierarchical Structures for High-Dimensional Time Series Forecastingcited by 5
  32. 2025BALM-TSF: Balanced Multimodal Alignment for LLM-Based Time Series Forecastingcited by 5
  33. 2025TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysiscited by 5
  34. 2025QuAnTS: Question Answering on Time Seriescited by 5
  35. 2025Using Causality for Enhanced Prediction of Web Traffic Time Seriescited by 4
  36. 2025GTM: A General Time-series Model for Enhanced Representation Learning of Time-Series Datacited by 4
  37. 2025Probabilistic Forecasting for Building Energy Systems using Time-Series Foundation Modelscited by 4
  38. 2025Pre-trained Large Language Models Learn Hidden Markov Models In-contextcited by 4

50 most cited of 163 All 163 (JSON)