# Are Language Models Actually Useful for Time Series Forecasting?

arXiv: https://arxiv.org/abs/2406.16964 · HTML: https://mlsanity.com/paper/2406.16964/

## Key facts

- Cited by 168 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 22 Jun 2024.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [Monash Time Series Forecasting Archive](https://mlsanity.com/paper/2105.06643/) (2021)
- [FITS: Modeling Time Series with 10k Parameters](https://mlsanity.com/paper/2307.03756/) (2023)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting](https://mlsanity.com/paper/2210.08964/) (2022)
- [TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series](https://mlsanity.com/paper/2308.08241/) (2023)
- [Large Language Models are Few-Shot Health Learners](https://mlsanity.com/paper/2305.15525/) (2023)
- [CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning](https://mlsanity.com/paper/2403.07300/) (2024)
- [Language Models Still Struggle to Zero-shot Reason about Time Series](https://mlsanity.com/paper/2404.11757/) (2024)
- [Time Series Prediction using Deep Learning Methods in Healthcare](https://mlsanity.com/paper/2108.13461/) (2021)
- [Learning to Embed Time Series Patches Independently](https://mlsanity.com/paper/2312.16427/) (2023)

## Built on by (most cited first, 30 of 168)

- [VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2408.17253/) (2024)
- [MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answering](https://mlsanity.com/paper/2503.16858/) (2025)
- [Can LLMs Understand Time Series Anomalies?](https://mlsanity.com/paper/2410.05440/) (2024)
- [Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative](https://mlsanity.com/paper/2502.08942/) (2025)
- [TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecaster](https://mlsanity.com/paper/2503.07649/) (2025)
- [SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognition](https://mlsanity.com/paper/2410.10624/) (2024)
- [Zero-shot forecasting of chaotic systems](https://mlsanity.com/paper/2409.15771/) (2024)
- [Harnessing Vision Models for Time Series Analysis: A Survey](https://mlsanity.com/paper/2502.08869/) (2025)
- [Specialized Foundation Models Struggle to Beat Supervised Baselines](https://mlsanity.com/paper/2411.02796/) (2024)
- [SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models](https://mlsanity.com/paper/2503.00211/) (2025)
- [FoNE: Precise Single-Token Number Embeddings via Fourier Features](https://mlsanity.com/paper/2502.09741/) (2025)
- [A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models](https://mlsanity.com/paper/2509.11575/) (2025)
- [True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics](https://mlsanity.com/paper/2505.13192/) (2025)
- [BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics](https://mlsanity.com/paper/2501.18972/) (2025)
- [CT-PatchTST: Channel-Time Patch Time-Series Transformer for Long-Term Renewable Energy Forecasting](https://mlsanity.com/paper/2501.08620/) (2025)
- [Out-of-Distribution Generalization in Time Series: A Survey](https://mlsanity.com/paper/2503.13868/) (2025)
- [SciTS: Scientific Time Series Understanding and Generation with LLMs](https://mlsanity.com/paper/2510.03255/) (2025)
- [Decision-informed Neural Networks with Large Language Model Integration for Portfolio Optimization](https://mlsanity.com/paper/2502.00828/) (2025)
- [Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting](https://mlsanity.com/paper/2602.17634/) (2026)
- [Breaking Silos: Adaptive Model Fusion Unlocks Better Time Series Forecasting](https://mlsanity.com/paper/2505.18442/) (2025)
- [BEDTime: A Unified Benchmark for Automatically Describing Time Series](https://mlsanity.com/paper/2509.05215/) (2025)
- [ViTime: Foundation Model for Time Series Forecasting Powered by Vision Intelligence](https://mlsanity.com/paper/2407.07311/) (2024)
- [A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions](https://mlsanity.com/paper/2502.06026/) (2025)
- [A Systematic Literature Review of Spatio-Temporal Graph Neural Network Models for Time Series Forecasting and Classification](https://mlsanity.com/paper/2410.22377/) (2024)
- [Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2505.24003/) (2025)
- [Financial Fine-tuning a Large Time Series Model](https://mlsanity.com/paper/2412.09880/) (2024)
- [There are no Champions in Supervised Long-Term Time Series Forecasting](https://mlsanity.com/paper/2502.14045/) (2025)
- [Understanding Why Large Language Models Can Be Ineffective in Time Series Analysis: The Impact of Modality Alignment](https://mlsanity.com/paper/2410.12326/) (2024)
- [TempoGPT: Enhancing Time Series Reasoning via Quantizing Embedding](https://mlsanity.com/paper/2501.07335/) (2025)
- [From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?](https://mlsanity.com/paper/2505.24030/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are being applied to time series forecasting. But are language models actually useful for time series? In a series of ablation studies on three recent and popular LLM-based time series forecasting methods, we find that removing the LLM component or replacing it with a basic attention layer does not degrade forecasting performance -- in most cases, the results even improve! We also find that despite their significant computational cost, pretrained LLMs do no better than models trained from scratch, do not represent the sequential dependencies in time series, and do not assist in few-shot settings. Additionally, we explore time series encoders and find that patching and attention structures perform similarly to LLM-based forecasters.