Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?
Not on the map. Why?
Abstract (arXiv)
The recent development of foundation models for time series data has generated considerable interest in using such models across a variety of applications. Although foundation models achieve state-of-the-art predictive performance, their calibration properties remain relatively underexplored, despite the fact that calibration can be critical for many practical applications. In this paper, we investigate the calibration-related properties of five recent time series foundation models and two competitive baselines. We perform a series of systematic evaluations assessing model calibration (i.e., over- or under-confidence), effects of varying prediction heads, and calibration under long-term autoregressive forecasting. We find that time series foundation models are consistently better calibrated than baseline models and tend not to be either systematically over- or under-confident, in contrast to the overconfidence often seen in other deep learning models.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 58 references that are in the map. First posted on arXiv 17 Oct 2025.
Builds on
28 of 58 references.
References · 28
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
- 2024Chronos: Learning the Language of Time Seriescited by 759
- 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
- 2023A decoder-only foundation model for time-series forecastingcited by 692
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
- 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
16 more
- 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
- 2024xLSTM: Extended Long Short-Term Memorycited by 327
- 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
- 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
- 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
- 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 120
- 2020Deep Learning for Time Series Forecasting: Tutorial and Literature Surveycited by 117
- 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 113
- 2019Distribution Calibration for Regressioncited by 89
- 2020Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantificationcited by 78
- 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 68
- 2022An Impartial Take to the CNN vs Transformer Robustness Contestcited by 34
- 2022UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Miningcited by 33
- 2023A Large-Scale Study of Probabilistic Calibration in Neural Network Regressioncited by 27
- 2025Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Modelcited by 24
- 2023Mitigating Transformer Overconfidence via Lipschitz Regularizationcited by 15
Built on by
Citing papers per year: 2026: 2.
2 papers.