# Robust Conformalized Selection with Noisy Responses

arXiv: https://arxiv.org/abs/2607.22985 · HTML: https://mlsanity.com/paper/2607.22985/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Builds on (most cited first, 11 of 11)

- [Theoretical Foundations of Conformal Prediction](https://mlsanity.com/paper/2411.11824/) (2024)
- [Training Convolutional Networks with Noisy Labels](https://mlsanity.com/paper/1406.2080/) (2014)
- [A Conformal Prediction Score that is Robust to Label Noise](https://mlsanity.com/paper/2405.02648/) (2024)
- [Conformal Selective Prediction with General Risk Control](https://mlsanity.com/paper/2603.24704/) (2026)
- [Probably Approximately Correct Labels](https://mlsanity.com/paper/2506.10908/) (2025)
- [Selection from Hierarchical Data with Conformal e-values](https://mlsanity.com/paper/2501.02514/) (2025)
- [Diversifying Conformal Selections](https://mlsanity.com/paper/2506.16229/) (2025)
- [ACS: An interactive framework for conformal selection](https://mlsanity.com/paper/2507.15825/) (2025)
- [A generalized e-value feature detection method with FDR control at multiple resolutions](https://mlsanity.com/paper/2409.17039/) (2024)
- [A Unified Framework for Large-Scale Inference of Classification: Error Rate Control and Optimality](https://mlsanity.com/paper/2504.07321/) (2025)
- [Feedback-Enhanced Online Multiple Testing with Applications to Conformal Selection](https://mlsanity.com/paper/2509.03297/) (2025)

## Abstract (arXiv)

Conformalized selection has been widely applied to select high-quality candidates from large datasets with rigorous uncertainty quantification, such as reliable labeling, drug discovery, and the alignment of large language models. Nevertheless, existing methods assume clean responses on calibration data, an assumption that rarely holds in practice. In this paper, we formulate the above tasks as selecting candidates with true predicted labels or with responses exceeding certain values. We demonstrate that existing conformal selection methods fail to control the false discovery rate (FDR) or suffer from severe power loss under contaminated calibration data. To that end, we propose Robust Conformalized Selection (RCS), a unified framework for selective classification with valid FDR control under general label contamination. The key insight of RCS lies in a novel statistical reduction: by separately conditioning on different classes, we translate the intractable label noise into a localized covariate shift problem, which then enables a covariate-adjusted empirical-Bayes-type estimate of the number of false selections. Statistical properties such as the asymptotic FDR control, power optimality, and robustness of RCS are established. We further develop an instantiation of RCS under randomized response model, and also apply RCS to the task of selecting candidates with large response values. Extensive experiments on both simulated and real-world datasets demonstrate the effectiveness of RCS.