# Adaptive conformal classification with noisy labels

arXiv: https://arxiv.org/abs/2309.05092 · HTML: https://mlsanity.com/paper/2309.05092/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 61 references that are in the map.
- First posted on arXiv 10 Sep 2023.

## Builds on (most cited first, 30 of 35)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Deep learning with noisy labels: exploring techniques and remedies in medical image analysis](https://mlsanity.com/paper/1912.02911/) (2019)
- [A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://mlsanity.com/paper/2010.12309/) (2020)
- [Conformal Inference of Counterfactuals and Individual Treatment Effects](https://mlsanity.com/paper/2006.06138/) (2020)
- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Conformal Inference for Online Prediction with Arbitrary Distribution Shifts](https://mlsanity.com/paper/2208.08401/) (2022)
- [Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey](https://mlsanity.com/paper/1912.05170/) (2019)
- [Conformal Prediction using Conditional Histograms](https://mlsanity.com/paper/2105.08747/) (2021)
- [A comparison of some conformal quantile regression methods](https://mlsanity.com/paper/1909.05433/) (2019)
- [Classification with Asymmetric Label Noise: Consistency and Maximal Denoising](https://mlsanity.com/paper/1303.1208/) (2013)
- [Training Uncertainty-Aware Classifiers with Conformalized Deep Learning](https://mlsanity.com/paper/2205.05878/) (2022)
- [Conformalized Survival Analysis](https://mlsanity.com/paper/2103.09763/) (2021)
- [Prediction and outlier detection in classification problems](https://mlsanity.com/paper/1905.04396/) (2019)
- [With Malice Towards None: Assessing Uncertainty via Equalized Coverage](https://mlsanity.com/paper/1908.05428/) (2019)
- [Conformal Sensitivity Analysis for Individual Treatment Effects](https://mlsanity.com/paper/2112.03493/) (2021)
- [Cautious Deep Learning](https://mlsanity.com/paper/1805.09460/) (2018)
- [Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers](https://mlsanity.com/paper/2208.11111/) (2022)
- [Derandomized Novelty Detection with FDR Control via Conformal E-values](https://mlsanity.com/paper/2302.07294/) (2023)

## Built on by (most cited first, 25 of 25)

- [Conformal Prediction for Deep Classifier via Label Ranking](https://mlsanity.com/paper/2310.06430/) (2023)
- [Conformal Prediction: A Data Perspective](https://mlsanity.com/paper/2410.06494/) (2024)
- [Wasserstein-regularized Conformal Prediction under General Distribution Shift](https://mlsanity.com/paper/2501.13430/) (2025)
- [Split Conformal Prediction under Data Contamination](https://mlsanity.com/paper/2407.07700/) (2024)
- [Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction](https://mlsanity.com/paper/2401.06421/) (2024)
- [Synthetic-Powered Predictive Inference](https://mlsanity.com/paper/2505.13432/) (2025)
- [Robust Conformal Prediction Using Privileged Information](https://mlsanity.com/paper/2406.05405/) (2024)
- [Conformal Prediction of Classifiers with Many Classes based on Noisy Labels](https://mlsanity.com/paper/2501.12749/) (2025)
- [Exploring the Noise Robustness of Online Conformal Prediction](https://mlsanity.com/paper/2501.18363/) (2025)
- [Calibrate, Don't Curate: Label-Efficient Estimation from Noisy LLM Judges](https://mlsanity.com/paper/2605.09702/) (2026)
- [Conformal Prediction with Corrupted Labels: Uncertain Imputation and Robust Re-weighting](https://mlsanity.com/paper/2505.04733/) (2025)
- [Meta-Analysis with Untrusted Data](https://mlsanity.com/paper/2407.09387/) (2024)
- [Privacy-Preserving Conformal Prediction Under Local Differential Privacy](https://mlsanity.com/paper/2505.15721/) (2025)
- [Efficient Conformal Prediction for Regression Models under Label Noise](https://mlsanity.com/paper/2509.15120/) (2025)
- [Elements of Conformal Prediction](https://mlsanity.com/paper/2603.23923/) (2026)
- [Weighted Conformal Prediction Provides Adaptive and Valid Mask-Conditional Coverage for General Missing Data Mechanisms](https://mlsanity.com/paper/2512.14221/) (2025)
- [Softmax is not Enough (for Adaptive Conformal Classification)](https://mlsanity.com/paper/2602.19498/) (2026)
- [Label Noise Cleaning for Supervised Classification via Bernoulli Random Sampling](https://mlsanity.com/paper/2603.14387/) (2026)
- [Robust Conditional Conformal Prediction via Branched Normalizing Flow](https://mlsanity.com/paper/2605.01868/) (2026)
- [Set-Valued Policy Learning](https://mlsanity.com/paper/2605.19830/) (2026)
- [Certified Domain Consistency for Multi-Domain Retrieval: Label-Free Per-Domain Contamination Control with Conformal Risk Guarantees](https://mlsanity.com/paper/2607.14157/) (2026)
- [Confidence Calibration of Deep Learning Systems](https://mlsanity.com/paper/2608.12100/) (2026)
- [The Exceedance Design Effect: Effective Sample Size for Thresholds under Clustering](https://mlsanity.com/paper/2608.21262/) (2026)
- [CHOIR: heterogeneity-aware conformal prediction for crash injury severity across driver safety strata](https://mlsanity.com/paper/2609.11592/) (2026)
- [Certified Against Which Oracle? Execution Labels Set the Reported Risk of Conformal Abstention for Text-to-SQL](https://mlsanity.com/paper/2609.25938/) (2026)

## Abstract (arXiv)

This paper develops novel conformal prediction methods for classification tasks that can automatically adapt to random label contamination in the calibration sample, leading to more informative prediction sets with stronger coverage guarantees compared to state-of-the-art approaches. This is made possible by a precise characterization of the effective coverage inflation (or deflation) suffered by standard conformal inferences in the presence of label contamination, which is then made actionable through new calibration algorithms. Our solution is flexible and can leverage different modeling assumptions about the label contamination process, while requiring no knowledge of the underlying data distribution or of the inner workings of the machine-learning classifier. The advantages of the proposed methods are demonstrated through extensive simulations and an application to object classification with the CIFAR-10H image data set.