When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters
Not on the map. Why?
Abstract (arXiv)
Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We provide the first systematic break-even analysis answering when this investment pays off. Across 30 benchmark datasets, we compare zero-shot and LoRA fine-tuned foundation models (Chronos, Moirai, Lag-Llama) against classical baselines (Naive, ETS, ARIMA, XGBoost) at six training set sizes from 2% to 100% of available data. Foundation models outperform classical methods at every evaluated training fraction on 15 of 30 datasets -- GPU deployment is unconditionally justified on these regardless of data volume. On 6 datasets, classical methods surpass zero-shot foundation models with as little as 2% of training data (21-2,768 samples); on the remaining 9, break-even ranges from 24 to 8,361 samples. One robust deployment rule requires no model training: if n_train < 700 and seasonality is non-negligible, use FM zero-shot and skip fine-tuning -- this resolves 10 of 30 deployment decisions immediately. Contrary to common practice, LoRA fine-tuning can actively degrade performance on short series. We operationalise these findings as a two-step decision framework -- compute dataset length and seasonality strength, run a brief 5-10% pilot only if needed -- enabling practitioners to make the FM-versus-classical decision before committing to full infrastructure. Four dataset features motivate mechanistic hypotheses for the remaining cases, though reliable automated prediction at this benchmark scale remains an open problem. Code, benchmark, and decision tools are available at https://github.com/nicolaisi/fm-breakeven.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 18 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
13 of 18 references.
References · 13
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2024Chronos: Learning the Language of Time Seriescited by 759
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
- 2021Monash Time Series Forecasting Archivecited by 267
- 2023FITS: Modeling Time Series with 10k Parameterscited by 195
- 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 113
- 2024Scaling Law for Time Series Forecastingcited by 33
- 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 24
- 2024Scaling-laws for Large Time-series Modelscited by 18
- 2025Efficient Model Selection for Time Series Forecasting via LLMscited by 4
1 more
- 2024AALF: Almost Always Linear Forecastingcited by 1
Built on by
Not cited yet.