# Post-selection Inference for Conformal Prediction: Trading off Coverage for Precision

arXiv: https://arxiv.org/abs/2304.06158 · HTML: https://mlsanity.com/paper/2304.06158/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 61 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 25 of 25)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Adaptive Conformal Predictions for Time Series](https://mlsanity.com/paper/2202.07282/) (2022)
- [Conformal Inference of Counterfactuals and Individual Treatment Effects](https://mlsanity.com/paper/2006.06138/) (2020)
- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Conformal Inference for Online Prediction with Arbitrary Distribution Shifts](https://mlsanity.com/paper/2208.08401/) (2022)
- [Distributional conformal prediction](https://mlsanity.com/paper/1909.07889/) (2019)
- [Improved Online Conformal Prediction via Strongly Adaptive Online Learning](https://mlsanity.com/paper/2302.07869/) (2023)
- [A comparison of some conformal quantile regression methods](https://mlsanity.com/paper/1909.05433/) (2019)
- [Selection by Prediction with Conformal p-values](https://mlsanity.com/paper/2210.01408/) (2022)
- [Adaptive, Distribution-Free Prediction Intervals for Deep Networks](https://mlsanity.com/paper/1905.10634/) (2019)
- [Conformalized Survival Analysis](https://mlsanity.com/paper/2103.09763/) (2021)
- [Doubly Robust Calibration of Prediction Sets under Covariate Shift](https://mlsanity.com/paper/2203.01761/) (2022)
- [Selecting the number of principal components: estimation of the true rank of a noisy matrix](https://mlsanity.com/paper/1410.8260/) (2014)
- [Conformalized survival analysis with adaptive cutoffs](https://mlsanity.com/paper/2211.01227/) (2022)
- [Conformal Prediction for Network-Assisted Regression](https://mlsanity.com/paper/2302.10095/) (2023)
- [Nested Conformal Prediction Sets for Classification with Applications to Probation Data](https://mlsanity.com/paper/2104.09358/) (2021)
- [Knowing what you know: valid and validated confidence sets in multiclass and multilabel prediction](https://mlsanity.com/paper/2004.10181/) (2020)
- [Exact Post-Selection Inference for Changepoint Detection and Other Generalized Lasso Problems](https://mlsanity.com/paper/1606.03552/) (2016)
- [The Essential Histogram](https://mlsanity.com/paper/1612.07216/) (2016)

## Built on by (most cited first, 5 of 5)

- [Transductive conformal inference with adaptive scores](https://mlsanity.com/paper/2310.18108/) (2023)
- [Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage](https://mlsanity.com/paper/2403.03868/) (2024)
- [Backward Conformal Prediction](https://mlsanity.com/paper/2505.13732/) (2025)
- [Data-Adaptive Tradeoffs among Multiple Risks in Distribution-Free Prediction](https://mlsanity.com/paper/2403.19605/) (2024)
- [Conformal Risk Sharing: Certified Cost Allocation with Participation Guarantees](https://mlsanity.com/paper/2606.06391/) (2026)

## Abstract (arXiv)

Conformal inference has played a pivotal role in providing uncertainty quantification for black-box ML prediction algorithms with finite sample guarantees. Traditionally, conformal prediction inference requires a data-independent specification of miscoverage level. In practical applications, one might want to update the miscoverage level after computing the prediction set. For example, in the context of binary classification, the analyst might start with a 95% prediction sets and see that most prediction sets contain all outcome classes. Prediction sets with both classes being undesirable, the analyst might desire to consider, say 80% prediction set. Construction of prediction sets that guarantee coverage with data-dependent miscoverage level can be considered as a post-selection inference problem. In this work, we develop simultaneous conformal inference to account for data-dependent miscoverage levels. Under the assumption of independent and identically distributed observations, our proposed methods have a finite sample simultaneous guarantee over all miscoverage levels. This allows practitioners to trade freely coverage probability for the quality of the prediction set by any criterion of their choice (say size of prediction set) while maintaining the finite sample guarantees similar to traditional conformal inference.