# Sequentially valid tests for forecast calibration

arXiv: https://arxiv.org/abs/2109.11761 · HTML: https://mlsanity.com/paper/2109.11761/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2021.

## Built on by (most cited first, 5 of 5)

- [Comparing Sequential Forecasters](https://mlsanity.com/paper/2110.00115/) (2021)
- [Combining Evidence Across Filtrations](https://mlsanity.com/paper/2402.09698/) (2024)
- [Online monotone density estimation and log-optimal calibration](https://mlsanity.com/paper/2602.08927/) (2026)
- [When Your Model Stops Working: Anytime-Valid Calibration Monitoring](https://mlsanity.com/paper/2603.13156/) (2026)
- [Bet on Features: Anytime-Valid and Feature-Aware Auditing of Conditional Quantile Forecasters](https://mlsanity.com/paper/2607.11653/) (2026)

## Abstract (arXiv)

Forecasting and forecast evaluation are inherently sequential tasks. Predictions are often issued on a regular basis, such as every hour, day, or month, and their quality is monitored continuously. However, the classical statistical tools for forecast evaluation are static, in the sense that statistical tests for forecast calibration are only valid if the evaluation period is fixed in advance. Recently, e-values have been introduced as a new, dynamic method for assessing statistical significance. An e-value is a non-negative random variable with expected value at most one under a null hypothesis. Large e-values give evidence against the null hypothesis, and the multiplicative inverse of an e-value is a conservative p-value. E-values are particularly suitable for sequential forecast evaluation, since they naturally lead to statistical tests which are valid under optional stopping. This article proposes e-values for testing probabilistic calibration of forecasts, which is one of the most important notions of calibration. The proposed methods are also more generally applicable for sequential goodness-of-fit testing. We demonstrate that the e-values are competitive in terms of power when compared to extant methods, which do not allow sequential testing. Furthermore, they provide important and useful insights in the evaluation of probabilistic weather forecasts.