mlsanityData to 14 Oct 2026

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting

arXiv 2310.08278 ↗First posted 12 Oct 2023cs.LG cs.AIcited by 113

Not on the map. Why?

Abstract (arXiv)

Over the past years, foundation models have caused a paradigm shift in machine learning due to their unprecedented capabilities for zero-shot and few-shot generalization. However, despite the success of foundation models in modalities such as natural language processing and computer vision, the development of foundation models for time series forecasting has lagged behind. We present Lag-Llama, a general-purpose foundation model for univariate probabilistic time series forecasting based on a decoder-only transformer architecture that uses lags as covariates. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities compared to a wide range of forecasting models on downstream datasets across domains. Moreover, when fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance, outperforming prior deep learning approaches, emerging as the best general-purpose model on average. Lag-Llama serves as a strong contender to the current state-of-art in time series forecasting and paves the way for future advancements in foundation models tailored to time series data.

Key facts. Cited by 113 papers in the map (data through 14 Oct 2026). Builds on 33 of its 66 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

33 of 66 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  9. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  10. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  11. 2019Root Mean Square Layer Normalizationcited by 1,480
  12. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
21 more
  1. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  2. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  3. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
  4. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  5. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 607
  6. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 391
  7. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 309
  8. 2021Monash Time Series Forecasting Archivecited by 267
  9. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  10. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 174
  11. 2019High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processescited by 148
  12. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 148
  13. 2020Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flowscited by 144
  14. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 144
  15. 2020Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Casecited by 129
  16. 2020Deep Learning for Time Series Forecasting: Tutorial and Literature Surveycited by 117
  17. 2022Broken Neural Scaling Lawscited by 109
  18. 2023Ti-MAE: Self-Supervised Masked Time Series Autoencoderscited by 63
  19. 2022TACTiS: Transformer-Attentional Copulas for Time Seriescited by 38
  20. 2013Highly comparative time-series analysis: The empirical structure of time series and their methodscited by 30
  21. 2023Mitigating Cold-start Forecasting using Cold Causal Demand Forecasting Modelcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 21; 2025: 30; 2026: 60.

113 papers.

Sort:

By area:

  1. 2024Chronos: Learning the Language of Time Seriescited by 759
  2. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 104
  3. 2025Moirai 2.0: When Less Is More for Time Series Forecastingcited by 69
  4. 2024Retrieval Augmented Time Series Forecastingcited by 46
  5. 2024BlackMamba: Mixture of Experts for State-Space Modelscited by 37
  6. 2023GATGPT: A Pre-trained Large Language Model with Graph Attention Network for Spatiotemporal Imputationcited by 35
  7. 2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 33
  8. 2024Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenizationcited by 27
  9. 2025MIRA: Medical Time Series Foundation Model for Real-World Health Datacited by 22
  10. 2025TimeSeriesScientist: A General-Purpose AI Agent for Time Series Analysiscited by 22
  11. 2024TimeRAF: Retrieval-Augmented Foundation model for Zero-shot Time Series Forecastingcited by 17
  12. 2024Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Datacited by 15
38 more
  1. 2025Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classificationcited by 9
  2. 2026Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Modelscited by 9
  3. 2024Macroeconomic Forecasting with Large Language Modelscited by 8
  4. 2024Efficient Time Series Processing for Transformers and State-Space Models through Token Mergingcited by 7
  5. 2025MoTime: A Dataset Suite for Multimodal Time Series Forecastingcited by 7
  6. 2026GlucoFM: A Dual-Stream Foundation Model for Continuous Glucose Monitoringcited by 7
  7. 2025Unlocking Pretrained Vision Transformers for Time Series Classificationcited by 6
  8. 2025Foundation models for time series forecasting: Application in conformal predictioncited by 6
  9. 2026Nexus : An Agentic Framework for Time Series Forecastingcited by 6
  10. 2026Towards a General Intelligence and Interface for Wearable Health Datacited by 6
  11. 2024Towards Long-Context Time Series Foundation Modelscited by 5
  12. 2025LAST SToP For Modeling Asynchronous Time Seriescited by 5
  13. 2025Byte Pair Encoding for Efficient Time Series Forecastingcited by 5
  14. 2026TempusBench: An Evaluation Framework for Time-Series Forecastingcited by 4
  15. 2026Adaptive Conformal Anomaly Detection with Time Series Foundation Models for Signal Monitoringcited by 4
  16. 2024The impact of data set similarity and diversity on transfer learning success in time series forecastingcited by 3
  17. 2024Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecastingcited by 3
  18. 2024Strada-LLM: Graph LLM for traffic predictioncited by 3
  19. 2024Dataset-Driven Channel Masks in Transformers for Multivariate Time Seriescited by 3
  20. 2025M2AD: Multi-Sensor Multi-System Anomaly Detection through Global Scoring and Calibrated Thresholdingcited by 3
  21. 2025Large Language models for Time Series Analysis: Techniques, Applications, and Challengescited by 3
  22. 2025RefineBridge: Generative Bridge Models Improve Financial Forecasting by Foundation Modelscited by 3
  23. 2026Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Surveycited by 3
  24. 2026Time Series Foundation Models as Strong Baselines in Transportation Forecasting: A Large-Scale Benchmark Analysiscited by 3
  25. 2026FactoryNet: A Large-Scale Dataset toward Industrial Time-Series Foundation Modelscited by 3
  26. 2026Bridging the Last Mile of Time Series Forecasting with LLM Agentscited by 3
  27. 2026EXAONE Finance 1.0: An Attention-free Time Series Foundation Model for Financial Time Seriescited by 3
  28. 2024LTSM-Bundle: A Toolbox and Benchmark on Large Language Models for Time Series Forecastingcited by 2
  29. 2024Partial-Multivariate Model for Forecastingcited by 2
  30. 2024Energy Price Modelling: A Comparative Evaluation of four Generations of Forecasting Methodscited by 2
  31. 2025Bridging Distribution Gaps in Time Series Foundation Model Pretraining with Prototype-Guided Normalizationcited by 2
  32. 2025Foundation Models Knowledge Distillation For Battery Capacity Degradation Forecastcited by 2
  33. 2025Towards a Foundation Model for Communication Systemscited by 2
  34. 2025Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?cited by 2
  35. 2026Time Series Foundation Models for Energy Load Forecasting on Consumer Hardware: A Multi-Dimensional Zero-Shot Benchmarkcited by 2
  36. 2026Time-Aware Prior Fitted Networks for Zero-Shot Forecasting with Exogenous Variablescited by 2
  37. 2026Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approachcited by 2
  38. 2026TimeClaw: A Time-Series AI Agent with Exploratory Execution Learningcited by 2

50 most cited of 113 All 113 (JSON)