# In-Context Fine-Tuning for Time-Series Foundation Models

arXiv: https://arxiv.org/abs/2410.24087 · HTML: https://mlsanity.com/paper/2410.24087/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 52 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [MetaICL: Learning to Learn In Context](https://mlsanity.com/paper/2110.15943/) (2021)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [Generating Wikipedia by Summarizing Long Sequences](https://mlsanity.com/paper/1801.10198/) (2018)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [Transformers learn to implement preconditioned gradient descent for in-context learning](https://mlsanity.com/paper/2306.00297/) (2023)
- [Monash Time Series Forecasting Archive](https://mlsanity.com/paper/2105.06643/) (2021)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [Meta-learning via Language Model In-context Tuning](https://mlsanity.com/paper/2110.07814/) (2021)

## Built on by (most cited first, 30 of 35)

- [Chronos-2: From Univariate to Universal Forecasting](https://mlsanity.com/paper/2510.15821/) (2025)
- [Timer-S1: A Billion-Scale Time Series Foundation Model with Serial Scaling](https://mlsanity.com/paper/2603.04791/) (2026)
- [Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challenges](https://mlsanity.com/paper/2510.13654/) (2025)
- [TiRex-2: Generalizing TiRex to Multivariate Data and Streaming](https://mlsanity.com/paper/2607.01204/) (2026)
- [TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis](https://mlsanity.com/paper/2510.06063/) (2025)
- [Universal Redundancies in Time Series Foundation Models](https://mlsanity.com/paper/2602.01605/) (2026)
- [Agent Bain vs. Agent McKinsey: A New Text-to-SQL Benchmark for the Business Domain](https://mlsanity.com/paper/2510.07309/) (2025)
- [Semantic Self-Distillation for Language Model Uncertainty](https://mlsanity.com/paper/2602.04577/) (2026)
- [Rethinking Zero-Shot Time Series Classification: From Task-specific Classifiers to In-Context Inference](https://mlsanity.com/paper/2602.00620/) (2026)
- [Why Do Time Series Models Need Long Context Windows?](https://mlsanity.com/paper/2606.01999/) (2026)
- [Post-Training in Time Series Foundation Models: A Unifying Framework](https://mlsanity.com/paper/2607.20002/) (2026)
- [The language of time: a language model perspective on time-series foundation models](https://mlsanity.com/paper/2507.00078/) (2025)
- [Position: The Inevitable End of One-Architecture-Fits-All-Domains in Time Series Forecasting](https://mlsanity.com/paper/2602.01736/) (2026)
- [Adapting Time Series Foundation Models through Data Mixtures](https://mlsanity.com/paper/2603.02840/) (2026)
- [A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks](https://mlsanity.com/paper/2603.22586/) (2026)
- [t0: A Time-Series Foundation Model for Forecasting with Context](https://mlsanity.com/paper/2609.24559/) (2026)
- [Time Series Foundation Models for Process Model Forecasting](https://mlsanity.com/paper/2512.07624/) (2025)
- [Enhancing Large Language Models for Time-Series Forecasting via Vector-Injected In-Context Learning](https://mlsanity.com/paper/2601.07903/) (2026)
- [TimeRadar: A Domain-Rotatable Foundation Model for Time Series Anomaly Detection](https://mlsanity.com/paper/2602.19068/) (2026)
- [GITCO: Gated Inference-Time Context Optimization in TSFMs](https://mlsanity.com/paper/2606.05332/) (2026)
- [Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamics](https://mlsanity.com/paper/2508.11680/) (2025)
- [Foundation Model Forecasts: Form and Function](https://mlsanity.com/paper/2510.19345/) (2025)
- [DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streams](https://mlsanity.com/paper/2511.17693/) (2025)
- [Kinematic Tokenization: Optimization-Based Continuous-Time Tokens for Learnable Decision Policies in Noisy Time Series](https://mlsanity.com/paper/2601.09949/) (2026)
- [Exploring the Potential of Probabilistic Transformer for Time Series Modeling: A Report on the ST-PT Framework](https://mlsanity.com/paper/2604.26762/) (2026)
- [On the Role of Inductive Bias in Time-Series Pretraining: A Case Study in Learning Generalizable Representations for Clinical Time Series](https://mlsanity.com/paper/2605.26194/) (2026)
- [RAID: Semantic Graph Diffusion for True Cold-Start and Cross-Lingual Forecasting](https://mlsanity.com/paper/2606.16925/) (2026)
- [Align-RAG: Alignment Is All You Need for TSFM In-Context Learning](https://mlsanity.com/paper/2608.05571/) (2026)
- [Beyond Average Error through Oracle-Informed Stress Tests for Time-Series Forecasting](https://mlsanity.com/paper/2609.22820/) (2026)
- [Instance-Adaptive Prompts as Context for Time-Series Foundation Models](https://mlsanity.com/paper/2609.34786/) (2026)

## Abstract (arXiv)

Motivated by the recent success of time-series foundation models for zero-shot forecasting, we present a methodology for textitin-context fine-tuning of a time-series foundation model. In particular, we design a pretrained foundation model that can be prompted (at inference time) with multiple time-series examples, in order to forecast a target time-series into the future. Our foundation model is specifically trained to utilize examples from multiple related time-series in its context window (in addition to the history of the target time-series) to help it adapt to the specific distribution of the target domain at inference time. We show that such a foundation model that uses in-context examples at inference time can obtain much better performance on popular forecasting benchmarks compared to supervised deep learning methods, statistical models, as well as other time-series foundation models. Interestingly, our in-context fine-tuning approach even rivals the performance of a foundation model that is explicitly fine-tuned on the target domain.