mlsanityData to 14 Oct 2026

In-Context Fine-Tuning for Time-Series Foundation Models

arXiv 2410.24087 ↗First posted 31 Oct 2024cs.LG cs.AI cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

Motivated by the recent success of time-series foundation models for zero-shot forecasting, we present a methodology for textitin-context fine-tuning of a time-series foundation model. In particular, we design a pretrained foundation model that can be prompted (at inference time) with multiple time-series examples, in order to forecast a target time-series into the future. Our foundation model is specifically trained to utilize examples from multiple related time-series in its context window (in addition to the history of the target time-series) to help it adapt to the specific distribution of the target domain at inference time. We show that such a foundation model that uses in-context examples at inference time can obtain much better performance on popular forecasting benchmarks compared to supervised deep learning methods, statistical models, as well as other time-series foundation models. Interestingly, our in-context fine-tuning approach even rivals the performance of a foundation model that is explicitly fine-tuned on the target domain.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

37 of 52 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  6. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  7. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  9. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  10. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  11. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  12. 2024Chronos: Learning the Language of Time Seriescited by 759
25 more
  1. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
  2. 2022Transformers learn in-context by gradient descentcited by 732
  3. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  4. 2023A decoder-only foundation model for time-series forecastingcited by 692
  5. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
  6. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  7. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  8. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  9. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  10. 2020Time Series Forecasting With Deep Learning: A Surveycited by 459
  11. 2021MetaICL: Learning to Learn In Contextcited by 454
  12. 2023Larger language models do in-context learning differentlycited by 394
  13. 2018Generating Wikipedia by Summarizing Long Sequencescited by 335
  14. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 332
  15. 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 279
  16. 2021Monash Time Series Forecasting Archivecited by 267
  17. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  18. 2021Meta-learning via Language Model In-context Tuningcited by 164
  19. 2019Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecastingcited by 161
  20. 2023TimeGPT-1cited by 158
  21. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 150
  22. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 104
  23. 2023In-context Pretraining: Language Modeling Beyond Document Boundariescited by 79
  24. 2023LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Predictioncited by 20
  25. 2014An Empirical Evaluation of Similarity Measures for Time Series Classificationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 26.

35 papers.

Sort:

By area:

  1. 2025Chronos-2: From Univariate to Universal Forecastingcited by 194
  2. 2026Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scalingcited by 28
  3. 2025Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challengescited by 16
  4. 2026TiRex-2: Generalizing TiRex to Multivariate Data and Streamingcited by 13
  5. 2025TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysiscited by 5
  6. 2026Universal Redundancies in Time Series Foundation Modelscited by 5
  7. 2025Agent Bain vs. Agent McKinsey: A New Text-to-SQL Benchmark for the Business Domaincited by 4
  8. 2026Semantic Self-Distillation for Language Model Uncertaintycited by 4
  9. 2026Rethinking Zero-Shot Time Series Classification: From Task-specific Classifiers to In-Context Inferencecited by 3
  10. 2026Why Do Time Series Models Need Long Context Windows?cited by 3
  11. 2026Post-Training in Time Series Foundation Models: A Unifying Frameworkcited by 3
  12. 2025The language of time: a language model perspective on time-series foundation modelscited by 2
23 more
  1. 2026Position: The Inevitable End of One-Architecture-Fits-All-Domains in Time Series Forecastingcited by 2
  2. 2026Adapting Time Series Foundation Models through Data Mixturescited by 2
  3. 2026A Foundation Model for Instruction-Conditioned In-Context Time Series Taskscited by 2
  4. 2026t0: A Time-Series Foundation Model for Forecasting with Contextcited by 2
  5. 2025Time Series Foundation Models for Process Model Forecastingcited by 1
  6. 2026Enhancing Large Language Models for Time-Series Forecasting via Vector-Injected In-Context Learningcited by 1
  7. 2026TimeRadar: A Domain-Rotatable Foundation Model for Time Series Anomaly Detectioncited by 1
  8. 2026GITCO: Gated Inference-Time Context Optimization in TSFMscited by 1
  9. 2025Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamicscited by 0
  10. 2025Foundation Model Forecasts: Form and Functioncited by 0
  11. 2025DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streamscited by 0
  12. 2026Kinematic Tokenization: Optimization-Based Continuous-Time Tokens for Learnable Decision Policies in Noisy Time Seriescited by 0
  13. 2026Exploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Frameworkcited by 0
  14. 2026On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Seriescited by 0
  15. 2026RAID: Semantic Graph Diffusion for True Cold-Start and Cross-Lingual Forecastingcited by 0
  16. 2026Align-RAG: Alignment Is All You Need for TSFM In-Context Learningcited by 0
  17. 2026Beyond Average Error through Oracle-Informed Stress Tests for Time-Series Forecastingcited by 0
  18. 2026Instance-Adaptive Prompts as Context for Time-Series Foundation Modelscited by 0
  19. 2026Volatility-Clustering Adaptation for Financial Time Seriescited by 0
  20. 2026Raw-Routed Mixture of Adapters: A Causal Intervention for Routing Collapse in Time Series Foundation Modelscited by 0
  21. 2026A Unified Scaling Law for Time Series Foundation Modelscited by 0
  22. 2026Anlu: Enabling In-Context Time Series Anomaly Detection in Foundation Models via Counterfactual Supervisioncited by 0
  23. 2026ECHO: Correcting Time Series Foundation Model Forecasts with Historical Analogues and Errorscited by 0