# Conformal Prediction With Conditional Guarantees

arXiv: https://arxiv.org/abs/2305.12616 · HTML: https://mlsanity.com/paper/2305.12616/

## Key facts

- Cited by 152 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2023.

## Built on by (most cited first, 30 of 152)

- [Language Models with Conformal Factuality Guarantees](https://mlsanity.com/paper/2402.10978/) (2024)
- [Class-Conditional Conformal Prediction with Many Classes](https://mlsanity.com/paper/2306.09335/) (2023)
- [Large language model validity via enhanced conformal prediction methods](https://mlsanity.com/paper/2406.09714/) (2024)
- [Conformal Prediction for Deep Classifier via Label Ranking](https://mlsanity.com/paper/2310.06430/) (2023)
- [Conformal Prediction for Natural Language Processing: A Survey](https://mlsanity.com/paper/2405.01976/) (2024)
- [Multicalibration for Confidence Scoring in LLMs](https://mlsanity.com/paper/2404.04689/) (2024)
- [High-Dimensional Prediction for Sequential Decision Making](https://mlsanity.com/paper/2310.17651/) (2023)
- [Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents](https://mlsanity.com/paper/2502.02561/) (2025)
- [Boosted Conformal Prediction Intervals](https://mlsanity.com/paper/2406.07449/) (2024)
- [Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)](https://mlsanity.com/paper/2405.06627/) (2024)
- [E-Values Expand the Scope of Conformal Prediction](https://mlsanity.com/paper/2503.13050/) (2025)
- [Optimal Transport-based Conformal Prediction](https://mlsanity.com/paper/2501.18991/) (2025)
- [Conformal Prediction for Causal Effects of Continuous Treatments](https://mlsanity.com/paper/2407.03094/) (2024)
- [End-to-end Conditional Robust Optimization](https://mlsanity.com/paper/2403.04670/) (2024)
- [Localized Adaptive Risk Control](https://mlsanity.com/paper/2405.07976/) (2024)
- [Rectifying Conformity Scores for Better Conditional Coverage](https://mlsanity.com/paper/2502.16336/) (2025)
- [Self-Calibrating Conformal Prediction](https://mlsanity.com/paper/2402.07307/) (2024)
- [Automatically Adaptive Conformal Risk Control](https://mlsanity.com/paper/2406.17819/) (2024)
- [Kernel-based Optimally Weighted Conformal Time-Series Prediction](https://mlsanity.com/paper/2405.16828/) (2024)
- [Wasserstein-regularized Conformal Prediction under General Distribution Shift](https://mlsanity.com/paper/2501.13430/) (2025)
- [Neural Conditional Probability for Uncertainty Quantification](https://mlsanity.com/paper/2407.01171/) (2024)
- [Conformal Prediction Sets Can Cause Disparate Impact](https://mlsanity.com/paper/2410.01888/) (2024)
- [Conformal Prediction for Image Segmentation Using Morphological Prediction Sets](https://mlsanity.com/paper/2503.05618/) (2025)
- [On Temperature Scaling and Conformal Prediction of Deep Classifiers](https://mlsanity.com/paper/2402.05806/) (2024)
- [Multi-group Uncertainty Quantification for Long-form Text Generation](https://mlsanity.com/paper/2407.21057/) (2024)
- [Semi-Supervised Risk Control via Prediction-Powered Inference](https://mlsanity.com/paper/2412.11174/) (2024)
- [The Relationship between No-Regret Learning and Online Conformal Prediction](https://mlsanity.com/paper/2502.10947/) (2025)
- [Kandinsky Conformal Prediction: Beyond Class- and Covariate-Conditional Coverage](https://mlsanity.com/paper/2502.17264/) (2025)
- [Does confidence calibration improve conformal prediction?](https://mlsanity.com/paper/2402.04344/) (2024)
- [Split Conformal Prediction under Data Contamination](https://mlsanity.com/paper/2407.07700/) (2024)

## Abstract (arXiv)

We consider the problem of constructing distribution-free prediction sets with finite-sample conditional guarantees. Prior work has shown that it is impossible to provide exact conditional coverage universally in finite samples. Thus, most popular methods only guarantee marginal coverage over the covariates or are restricted to a limited set of conditional targets, e.g. coverage over a finite set of pre-specified subgroups. This paper bridges this gap by defining a spectrum of problems that interpolate between marginal and conditional validity. We motivate these problems by reformulating conditional coverage as coverage over a class of covariate shifts. When the target class of shifts is finite-dimensional, we show how to simultaneously obtain exact finite-sample coverage over all possible shifts. For example, given a collection of subgroups, our prediction sets guarantee coverage over each group. For more flexible, infinite-dimensional classes where exact coverage is impossible, we provide a procedure for quantifying the coverage errors of our algorithm. Moreover, by tuning interpretable hyperparameters, we allow the practitioner to control the size of these errors across shifts of interest. Our methods can be incorporated into existing split conformal inference pipelines, and thus can be used to quantify the uncertainty of modern black-box algorithms without distributional assumptions.