# Anytime-Valid Confirmation of Label-Shift Corrections

arXiv: https://arxiv.org/abs/2606.14028 · HTML: https://mlsanity.com/paper/2606.14028/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 12 Jun 2026.

## Builds on (most cited first, 9 of 9)

- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Game-theoretic statistics and safe anytime-valid inference](https://mlsanity.com/paper/2210.01948/) (2022)
- [A Unified View of Label Shift Estimation](https://mlsanity.com/paper/2003.07554/) (2020)
- [Safe Testing](https://mlsanity.com/paper/1906.07801/) (2019)
- [Distribution-free uncertainty quantification for classification under label shift](https://mlsanity.com/paper/2103.03323/) (2021)
- [Sequential Predictive Conformal Inference for Time Series](https://mlsanity.com/paper/2212.03463/) (2022)
- [There are natural scores: Full comment on Shafer, "Testing by betting: A strategy for statistical and scientific communication"](https://mlsanity.com/paper/2102.05569/) (2021)
- [Conformal Bayes under Label Shift: Post-Hoc Calibration vs. In-Training Adaptation](https://mlsanity.com/paper/2606.11865/) (2026)

## Built on by (most cited first, 1 of 1)

- [Anytime-Valid Confirmation of Covariate Balance for Prespecified Corrections](https://mlsanity.com/paper/2607.23157/) (2026)

## Abstract (arXiv)

In small-batch scientific deployments, labeled target outcomes may be too scarce for reliable shift estimation even when unlabeled target inputs are available. We address the complementary setting where the practitioner has a pre-specified label-shift correction from domain knowledge and asks whether incoming labeled outcomes support it. We show that the per-observation likelihood ratio between a label-shift-corrected predictive and the source predictive is a conditional e-value, so its running product is a nonnegative martingale and Ville's inequality yields an anytime-valid confirmation rule. The log martingale equals the cumulative negative log-predictive density (NLPD) gap between the source and the corrected predictive, converting routine model monitoring into a formal sequential test. Rejection means the incoming data support the posited correction relative to the source predictive, but it is not a precise estimate of the degree of shift. Closed forms are available for GP sources with Gaussian label-shift ratios. GP regression simulations validate Type I control, finite-sample power, miscalibration sensitivity, and the small-batch advantage of a reliable prior over label-based re-estimation.