# Selection by Prediction with Conformal p-values

arXiv: https://arxiv.org/abs/2210.01408 · HTML: https://mlsanity.com/paper/2210.01408/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 74 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Conformal Inference of Counterfactuals and Individual Treatment Effects](https://mlsanity.com/paper/2006.06138/) (2020)
- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Distributional conformal prediction](https://mlsanity.com/paper/1909.07889/) (2019)
- [End-to-end Training for Whole Image Breast Cancer Diagnosis using An All Convolutional Design](https://mlsanity.com/paper/1711.05775/) (2017)
- [Sensitivity Analysis of Individual Treatment Effects: A Robust Conformal Inference Approach](https://mlsanity.com/paper/2111.12161/) (2021)
- [Conformalized Survival Analysis](https://mlsanity.com/paper/2103.09763/) (2021)
- [Improving Screening Processes via Calibrated Subset Selection](https://mlsanity.com/paper/2202.01147/) (2022)
- [Concepts and Applications of Conformal Prediction in Computational Drug Discovery](https://mlsanity.com/paper/1908.03569/) (2019)
- [Online Control of the False Coverage Rate and False Sign Rate](https://mlsanity.com/paper/1905.01059/) (2019)
- [Semi-supervised multiple testing](https://mlsanity.com/paper/2106.13501/) (2021)

## Built on by (most cited first, 30 of 69)

- [Uncertainty Quantification over Graph with Conformalized Graph Neural Networks](https://mlsanity.com/paper/2305.14535/) (2023)
- [Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees](https://mlsanity.com/paper/2405.10301/) (2024)
- [Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)](https://mlsanity.com/paper/2405.06627/) (2024)
- [E-Values Expand the Scope of Conformal Prediction](https://mlsanity.com/paper/2503.13050/) (2025)
- [Selective Generation for Controllable Language Models](https://mlsanity.com/paper/2307.09254/) (2023)
- [Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage](https://mlsanity.com/paper/2403.03868/) (2024)
- [Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimization](https://mlsanity.com/paper/2411.17983/) (2024)
- [Conformal link prediction for false discovery rate control](https://mlsanity.com/paper/2306.14693/) (2023)
- [Conformalized Link Prediction on Graph Neural Networks](https://mlsanity.com/paper/2406.18763/) (2024)
- [Conformalized Interval Arithmetic with Symmetric Calibration](https://mlsanity.com/paper/2408.10939/) (2024)
- [Multivariate Conformal Selection](https://mlsanity.com/paper/2505.00917/) (2025)
- [Max-Rank: Efficient Multiple Testing for Conformal Prediction](https://mlsanity.com/paper/2311.10900/) (2023)
- [Selecting informative conformal prediction sets with false coverage rate control](https://mlsanity.com/paper/2403.12295/) (2024)
- [LEC: Linear Expectation Constraints for Selection-Conditioned Risk Control in Selective Prediction and Routing Systems](https://mlsanity.com/paper/2512.01556/) (2025)
- [PASC: Pipeline-Aware Conformal Prediction with Joint Coverage Guarantees for Multi-Stage NLP and LLM Pipelines](https://mlsanity.com/paper/2605.18812/) (2026)
- [Conformal Policy Control](https://mlsanity.com/paper/2603.02196/) (2026)
- [Post-selection Inference for Conformal Prediction: Trading off Coverage for Precision](https://mlsanity.com/paper/2304.06158/) (2023)
- [Conformalizing Machine Translation Evaluation](https://mlsanity.com/paper/2306.06221/) (2023)
- [Conformalized Deep Splines for Optimal and Efficient Prediction Sets](https://mlsanity.com/paper/2311.00774/) (2023)
- [Valid Selection among Conformal Sets](https://mlsanity.com/paper/2506.20173/) (2025)
- [Theoretical guarantees for change localization using conformal p-values](https://mlsanity.com/paper/2510.08749/) (2025)
- [Conditional Testing based on Localized Conformal p-values](https://mlsanity.com/paper/2409.16829/) (2024)
- [Multi-Condition Conformal Selection](https://mlsanity.com/paper/2510.08075/) (2025)
- [Symmetric Aggregation of Conformity Scores for Efficient Uncertainty Sets](https://mlsanity.com/paper/2512.06945/) (2025)
- [Conditional Prediction ROC Bands for Graph Classification](https://mlsanity.com/paper/2410.15239/) (2024)
- [Leave-One-Out Stable Conformal Prediction](https://mlsanity.com/paper/2504.12189/) (2025)
- [Online Conformal Selection with Accept-to-Reject Changes](https://mlsanity.com/paper/2508.13838/) (2025)
- [ATTS: Asynchronous Test-Time Scaling via Conformal Prediction](https://mlsanity.com/paper/2509.15148/) (2025)
- [Model-agnostic Selective Labeling with Provable Statistical Guarantees](https://mlsanity.com/paper/2510.14581/) (2025)
- [On the Within-Group Fairness of Screening Classifiers](https://mlsanity.com/paper/2302.00025/) (2023)

## Abstract (arXiv)

Decision making or scientific discovery pipelines such as job hiring and drug discovery often involve multiple stages: before any resource-intensive step, there is often an initial screening that uses predictions from a machine learning model to shortlist a few candidates from a large pool. We study screening procedures that aim to select candidates whose unobserved outcomes exceed user-specified values. We develop a method that wraps around any prediction model to produce a subset of candidates while controlling the proportion of falsely selected units. Building upon the conformal inference framework, our method first constructs p-values that quantify the statistical evidence for large outcomes; it then determines the shortlist by comparing the p-values to a threshold introduced in the multiple testing literature. In many cases, the procedure selects candidates whose predictions are above a data-dependent threshold. Our theoretical guarantee holds under mild exchangeability conditions on the samples, generalizing existing results on multiple conformal p-values. We demonstrate the empirical performance of our method via simulations, and apply it to job hiring and drug discovery datasets.