Adapting Time Series Foundation Models through Data Mixtures
Not on the map. Why?
Abstract (arXiv)
Time series foundation models (TSFMs) have become increasingly popular for zero-shot forecasting. However, for a new time series domain not fully covered by the pretraining set, performance can suffer. Therefore, when a practitioner cares about a new domain and has access to a set of related datasets, the question arises: how best to fine-tune a TSFM to improve zero-shot forecasting? A typical approach to this type of problem is to fine-tune a LoRA module on all datasets or separately on each dataset. Tuning a separate module on each dataset allows for the specialisation of the TSFM to different types of data distribution, by selecting differing combinations of per-dataset modules for different time series contexts. However, we find that, using per-dataset modules might not be optimal, since a time series dataset can contain data from several types of distributions, i.e. sub-domains. This can be due to the distribution shifting or having differing distributions for different dimensions of the time series. Hence, we propose MixFT which re-divides the data using Bayesian mixtures into sets that best represent the sub-domains present in the data, and fine-tunes separately on each of these sets. This re-division of the data ensures that each set is more homogeneous, leading to fine-tuned modules focused on specific sub-domains. Our experiments show that MixFT performs better than per-dataset methods and when fine-tuning a single module on all the data. This suggests that by re-partitioning the data to represent sub-domains we can better specialise TSFMs to improve zero-shot forecasting.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 3 Mar 2026.
Builds on
39 of 57 references.
References · 39
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
- 2024Chronos: Learning the Language of Time Seriescited by 712
- 2023A decoder-only foundation model for time-series forecastingcited by 650
- 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
- 2023DataComp: In search of the next generation of multimodal datasetscited by 635
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
- 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
- 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
- 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
27 more
- 2021Monash Time Series Forecasting Archivecited by 256
- 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
- 2019Topic Modeling in Embedding Spacescited by 224
- 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 188
- 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 120
- 2023AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Modelscited by 84
- 2024Towards Modular LLMs by Building and Reusing a Library of LoRAscited by 72
- 2022Continual Learning with Foundation Models: An Empirical Study of Latent Replaycited by 61
- 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
- 2024An Analysis of Linear Time Series Forecasting Modelscited by 40
- 2025TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecastercited by 36
- 2022Multi-Head Adapter Routing for Cross-Task Generalizationcited by 33
- 2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 32
- 2025Battling the Non-stationarity in Time Series Forecasting via Test-time Adaptationcited by 32
- 2024In-Context Fine-Tuning for Time-Series Foundation Modelscited by 31
- 2024Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecastingcited by 18
- 2023AutoMixer for Improved Multivariate Time-Series Forecasting on Business and IT Observability Datacited by 17
- 2025Lightweight Online Adaption for Time Series Foundation Model Forecastscited by 14
- 2024Addressing Concept Shift in Online Time Series Forecasting: Detect-then-Adaptcited by 11
- 2022On the Limitations of General Purpose Domain Generalisation Methodscited by 10
- 2024DAM: Towards A Foundation Model for Time Series Forecastingcited by 10
- 2024Beyond LoRA: Exploring Efficient Fine-Tuning Techniques for Time Series Foundational Modelscited by 10
- 2025Performance of Zero-Shot Time Series Foundation Models on Cloud Datacited by 10
- 2024Adapt-∞: Scalable Continual Multimodal Instruction Tuning via Dynamic Data Selectioncited by 9
- 2021Evaluating Privacy-Preserving Machine Learning in Critical Infrastructures: A Case Study on Time-Series Classificationcited by 6
- 2024The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuningcited by 4
- 2025A Generalization Theory for Zero-Shot Predictioncited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.