# Online Selective Conformal Prediction: Errors and Solutions

arXiv: https://arxiv.org/abs/2503.16809 · HTML: https://mlsanity.com/paper/2503.16809/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Conformal Inference for Online Prediction with Arbitrary Distribution Shifts](https://mlsanity.com/paper/2208.08401/) (2022)
- [Theoretical Foundations of Conformal Prediction](https://mlsanity.com/paper/2411.11824/) (2024)
- [Exact post-selection inference, with application to the lasso](https://mlsanity.com/paper/1311.6238/) (2013)
- [Selection by Prediction with Conformal p-values](https://mlsanity.com/paper/2210.01408/) (2022)
- [Optimal Inference After Model Selection](https://mlsanity.com/paper/1410.2597/) (2014)
- [Online conformal prediction with decaying step sizes](https://mlsanity.com/paper/2402.01139/) (2024)
- [Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimization](https://mlsanity.com/paper/2411.17983/) (2024)
- [Online Control of the False Coverage Rate and False Sign Rate](https://mlsanity.com/paper/1905.01059/) (2019)
- [Selective conformal inference with false coverage-statement rate control](https://mlsanity.com/paper/2301.00584/) (2023)
- [CAP: A General Algorithm for Online Selective Conformal Prediction with FCR Control](https://mlsanity.com/paper/2403.07728/) (2024)

## Built on by (most cited first, 2 of 2)

- [Online Selective Conformal Prediction with Asymmetric Rules: A Permutation Test Approach](https://mlsanity.com/paper/2602.10018/) (2026)
- [Counterfactual Online Conformal Prediction Under Adaptive Logging](https://mlsanity.com/paper/2609.30811/) (2026)

## Abstract (arXiv)

In online selective conformal inference, data arrives sequentially, and prediction intervals are constructed only when an online selection rule is met. Since online selections may break the exchangeability between the selected test datum and the rest of the data, one must correct for this by suitably selecting the calibration data. In this paper, we evaluate existing calibration selection strategies and pinpoint some fundamental errors in the associated claims that guarantee selection-conditional coverage and control of the false coverage rate (FCR). To address these shortcomings, we propose novel calibration selection strategies that provably preserve the exchangeability of the calibration data and the selected test datum. Consequently, we demonstrate that online selective conformal inference with these strategies guarantees both selection-conditional coverage and FCR control. Our theoretical findings are supported by experimental evidence examining tradeoffs between valid methods.