# Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?

arXiv: https://arxiv.org/abs/2510.16060 · HTML: https://mlsanity.com/paper/2510.16060/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 58 references that are in the map.
- First posted on arXiv 17 Oct 2025.

## Builds on (most cited first, 28 of 28)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [xLSTM: Extended Long Short-Term Memory](https://mlsanity.com/paper/2405.04517/) (2024)
- [Foundation Models for Time Series Analysis: A Tutorial and Survey](https://mlsanity.com/paper/2403.14735/) (2024)
- [GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation](https://mlsanity.com/paper/2410.10393/) (2024)
- [Reducing conversational agents' overconfidence through linguistic calibration](https://mlsanity.com/paper/2012.14983/) (2020)
- [TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning](https://mlsanity.com/paper/2505.23719/) (2025)
- [Deep Learning for Time Series Forecasting: Tutorial and Literature Survey](https://mlsanity.com/paper/2004.10240/) (2020)
- [Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2310.08278/) (2023)
- [Distribution Calibration for Regression](https://mlsanity.com/paper/1905.06023/) (2019)
- [Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification](https://mlsanity.com/paper/2011.09588/) (2020)
- [This Time is Different: An Observability Perspective on Time Series Foundation Models](https://mlsanity.com/paper/2505.14766/) (2025)
- [An Impartial Take to the CNN vs Transformer Robustness Contest](https://mlsanity.com/paper/2207.11347/) (2022)
- [UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining](https://mlsanity.com/paper/2202.13469/) (2022)
- [A Large-Scale Study of Probabilistic Calibration in Neural Network Regression](https://mlsanity.com/paper/2306.02738/) (2023)
- [Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model](https://mlsanity.com/paper/2506.11029/) (2025)
- [Mitigating Transformer Overconfidence via Lipschitz Regularization](https://mlsanity.com/paper/2306.06849/) (2023)

## Built on by (most cited first, 2 of 2)

- [Do Time Series Foundation Model Benchmarks Hide Regime-Dependent Failures? Evidence from Traffic Speed Forecasting](https://mlsanity.com/paper/2606.18367/) (2026)
- [SGA: Uncertainty Quantification for Multi-Step Forecasting in Time Series Foundation Models](https://mlsanity.com/paper/2609.28582/) (2026)

## Abstract (arXiv)

The recent development of foundation models for time series data has generated considerable interest in using such models across a variety of applications. Although foundation models achieve state-of-the-art predictive performance, their calibration properties remain relatively underexplored, despite the fact that calibration can be critical for many practical applications. In this paper, we investigate the calibration-related properties of five recent time series foundation models and two competitive baselines. We perform a series of systematic evaluations assessing model calibration (i.e., over- or under-confidence), effects of varying prediction heads, and calibration under long-term autoregressive forecasting. We find that time series foundation models are consistently better calibrated than baseline models and tend not to be either systematically over- or under-confident, in contrast to the overconfidence often seen in other deep learning models.