# Beyond Fixed False Discovery Rates: Post-Hoc Conformal Selection with E-Variables

arXiv: https://arxiv.org/abs/2604.11305 · HTML: https://mlsanity.com/paper/2604.11305/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 13 Apr 2026.

## Builds on (most cited first, 30 of 30)

- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Safe Testing](https://mlsanity.com/paper/1906.07801/) (2019)
- [Selection by Prediction with Conformal p-values](https://mlsanity.com/paper/2210.01408/) (2022)
- [Sensitivity Analysis of Individual Treatment Effects: A Robust Conformal Inference Approach](https://mlsanity.com/paper/2111.12161/) (2021)
- [Conformal prediction for time series](https://mlsanity.com/paper/2010.09107/) (2020)
- [Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2202.08756/) (2022)
- [E-Values Expand the Scope of Conformal Prediction](https://mlsanity.com/paper/2503.13050/) (2025)
- [Towards Human-AI Complementarity with Prediction Sets](https://mlsanity.com/paper/2405.17544/) (2024)
- [Transductive conformal inference with adaptive scores](https://mlsanity.com/paper/2310.18108/) (2023)
- [Enhancing Conformal Prediction Using E-Test Statistics](https://mlsanity.com/paper/2403.19082/) (2024)
- [Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction](https://mlsanity.com/paper/2210.03067/) (2022)
- [Derandomized Novelty Detection with FDR Control via Conformal E-values](https://mlsanity.com/paper/2302.07294/) (2023)
- [Post-selection inference for e-value based confidence intervals](https://mlsanity.com/paper/2203.12572/) (2022)
- [Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimization](https://mlsanity.com/paper/2411.17983/) (2024)
- [Backward Conformal Prediction](https://mlsanity.com/paper/2505.13732/) (2025)
- [False Discovery Rate Control via Data Splitting](https://mlsanity.com/paper/2002.08542/) (2020)
- [Conformal Selective Prediction with General Risk Control](https://mlsanity.com/paper/2603.24704/) (2026)
- [Multivariate Conformal Selection](https://mlsanity.com/paper/2505.00917/) (2025)
- [Selection from Hierarchical Data with Conformal e-values](https://mlsanity.com/paper/2501.02514/) (2025)
- [Diversifying Conformal Selections](https://mlsanity.com/paper/2506.16229/) (2025)
- [Multi-Condition Conformal Selection](https://mlsanity.com/paper/2510.08075/) (2025)
- [Permutation-based simultaneous confidence bounds for the false discovery proportion](https://mlsanity.com/paper/1808.05528/) (2018)
- [More powerful multiple testing under dependence via randomization](https://mlsanity.com/paper/2305.11126/) (2023)
- [Boosting e-BH via conditional calibration](https://mlsanity.com/paper/2404.17562/) (2024)
- [Asymptotic and compound e-values: multiple testing and empirical Bayes](https://mlsanity.com/paper/2409.19812/) (2024)
- [Revamping Conformal Selection With Optimal Power: A Neyman--Pearson Perspective](https://mlsanity.com/paper/2502.16513/) (2025)
- [Online Conformal Selection with Accept-to-Reject Changes](https://mlsanity.com/paper/2508.13838/) (2025)
- [Fuzzy Prediction Sets: Conformal Prediction with E-values](https://mlsanity.com/paper/2509.13130/) (2025)
- [Bringing Closure to False Discovery Rate Control: A General Principle for Multiple Testing](https://mlsanity.com/paper/2509.02517/) (2025)
- [Feedback-Enhanced Online Multiple Testing with Applications to Conformal Selection](https://mlsanity.com/paper/2509.03297/) (2025)

## Built on by (most cited first, 1 of 1)

- [Everywhere Valid Bounds on False Discovery Proportions in Conformal Inference](https://mlsanity.com/paper/2605.20726/) (2026)

## Abstract (arXiv)

Conformal selection (CS) uses calibration data to identify test inputs whose unobserved outcomes are likely to satisfy a pre-specified minimal quality requirement, while controlling the false discovery rate (FDR). Existing methods fix the target FDR level before observing data, which prevents the user from adapting the balance between number of selected test inputs and FDR to downstream needs and constraints based on the available data. For example, in genomics or neuroimaging, researchers often inspect the distribution of test statistics, and decide how aggressively to pursue candidates based on observed evidence strength and available follow-up resources. To address this limitation, we introduce post-hoc CS (PH-CS), which generates a path of candidate selection sets, each paired with a data-driven false discovery proportion (FDP) estimate. PH-CS lets the user select any operating point on this path by maximizing a user-specified utility, arbitrarily balancing selection size and FDR. Building on conformal e-variables and the e-Benjamini-Hochberg (e-BH) procedure, PH-CS is proved to provide a finite-sample post-hoc reliability guarantee whereby the ratio between estimated FDP level and true FDP is, on average, upper bounded by 1, so that the average estimated FDP is, to first order, a valid upper bound on the true FDR. PH-CS is extended to control quality defined in terms of a general risk. Experiments on synthetic and real-world datasets demonstrate that, unlike CS, PH-CS can consistently satisfy user-imposed utility constraints while producing reliable FDP estimates and maintaining competitive FDR control.