# How Foundational are Foundation Models for Time Series Forecasting?

arXiv: https://arxiv.org/abs/2510.00742 · HTML: https://mlsanity.com/paper/2510.00742/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 18 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [TimeGPT-1](https://mlsanity.com/paper/2310.03589/) (2023)
- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention](https://mlsanity.com/paper/2402.10198/) (2024)
- [Specialized Foundation Models Struggle to Beat Supervised Baselines](https://mlsanity.com/paper/2411.02796/) (2024)
- [Beyond One-Size-Fits-All: Tailored Benchmarks for Efficient Evaluation](https://mlsanity.com/paper/2502.13576/) (2025)

## Built on by (most cited first, 5 of 5)

- [Aionoscope: Debugging Latent-State Accessibility in Time-Series Representations](https://mlsanity.com/paper/2607.00956/) (2026)
- [Are Vision Foundation Models Foundational for Electron Microscopy Image Segmentation?](https://mlsanity.com/paper/2602.08505/) (2026)
- [Tracing the Evidence Behind Zero-Shot Time-Series Forecasting: A Source-First Taxonomy and Audit Framework](https://mlsanity.com/paper/2609.21425/) (2026)
- [Variational Streaming Flow: Probabilistic Forecasting in Physical Time](https://mlsanity.com/paper/2610.00976/) (2026)
- [Foundations without Fundamentals: Zero-Shot Blind Spots in Time Series FMs](https://mlsanity.com/paper/2610.02058/) (2026)

## Abstract (arXiv)

Foundation Models are designed to serve as versatile embedding machines, with strong zero shot capabilities and superior generalization performance when fine-tuned on diverse downstream tasks. While this is largely true for language and vision foundation models, we argue that the inherent diversity of time series data makes them less suited for building effective foundation models. We demonstrate this using forecasting as our downstream task. We show that the zero-shot capabilities of a time series foundation model are significantly influenced and tied to the specific domains it has been pretrained on. Furthermore, when applied to unseen real-world time series data, fine-tuned foundation models do not consistently yield substantially better results, relative to their increased parameter count and memory footprint, than smaller, dedicated models tailored to the specific forecasting task at hand.