# Conformal Risk Control

arXiv: https://arxiv.org/abs/2208.02814 · HTML: https://mlsanity.com/paper/2208.02814/

## Key facts

- Cited by 299 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [Conformal Inference of Counterfactuals and Individual Treatment Effects](https://mlsanity.com/paper/2006.06138/) (2020)
- [TResNet: High Performance GPU-Dedicated Architecture](https://mlsanity.com/paper/2003.13630/) (2020)
- [Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging](https://mlsanity.com/paper/2202.05265/) (2022)
- [Consistent Accelerated Inference via Confident Adaptive Transformers](https://mlsanity.com/paper/2104.08803/) (2021)
- [PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction](https://mlsanity.com/paper/2001.00106/) (2019)
- [Conformalized Survival Analysis](https://mlsanity.com/paper/2103.09763/) (2021)
- [Conformal Prediction Sets with Limited False Positives](https://mlsanity.com/paper/2202.07650/) (2022)
- [Conformal prediction with localization](https://mlsanity.com/paper/1908.08558/) (2019)
- [Semantic uncertainty intervals for disentangled latent spaces](https://mlsanity.com/paper/2207.10074/) (2022)
- [Recommendation Systems with Distribution-Free Reliability Guarantees](https://mlsanity.com/paper/2207.01609/) (2022)

## Built on by (most cited first, 30 of 299)

- [Conformal Language Modeling](https://mlsanity.com/paper/2306.10193/) (2023)
- [Language Models with Conformal Factuality Guarantees](https://mlsanity.com/paper/2402.10978/) (2024)
- [Large language model validity via enhanced conformal prediction methods](https://mlsanity.com/paper/2406.09714/) (2024)
- [Conformal PID Control for Time Series Prediction](https://mlsanity.com/paper/2307.16895/) (2023)
- [Improved Online Conformal Prediction via Strongly Adaptive Online Learning](https://mlsanity.com/paper/2302.07869/) (2023)
- [Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles](https://mlsanity.com/paper/2210.16955/) (2022)
- [Mitigating LLM Hallucinations via Conformal Abstention](https://mlsanity.com/paper/2405.01563/) (2024)
- [Conformal Prediction for Natural Language Processing: A Survey](https://mlsanity.com/paper/2405.01976/) (2024)
- [How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Control](https://mlsanity.com/paper/2302.03791/) (2023)
- [Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions](https://mlsanity.com/paper/2310.05921/) (2023)
- [Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents](https://mlsanity.com/paper/2502.02561/) (2025)
- [Calibrated Selective Classification](https://mlsanity.com/paper/2208.12084/) (2022)
- [Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagation](https://mlsanity.com/paper/2303.12246/) (2023)
- [Conformal Prediction Regions for Time Series using Linear Complementarity Programming](https://mlsanity.com/paper/2304.01075/) (2023)
- [Fast yet Safe: Early-Exiting with Risk Control](https://mlsanity.com/paper/2405.20915/) (2024)
- [Distribution Free Prediction Sets for Node Classification](https://mlsanity.com/paper/2211.14555/) (2022)
- [Principal Uncertainty Quantification with Spatial Correlation for Image Restoration Problems](https://mlsanity.com/paper/2305.10124/) (2023)
- [C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models](https://mlsanity.com/paper/2402.03181/) (2024)
- [Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)](https://mlsanity.com/paper/2405.06627/) (2024)
- [Conformal Prediction: A Data Perspective](https://mlsanity.com/paper/2410.06494/) (2024)
- [A Survey on the Honesty of Large Language Models](https://mlsanity.com/paper/2409.18786/) (2024)
- [Efficiently Controlling Multiple Risks with Pareto Testing](https://mlsanity.com/paper/2210.07913/) (2022)
- [Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling](https://mlsanity.com/paper/2304.06052/) (2023)
- [Mitigating Temporal Misalignment by Discarding Outdated Facts](https://mlsanity.com/paper/2305.14824/) (2023)
- [Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Control](https://mlsanity.com/paper/2310.10299/) (2023)
- [Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty](https://mlsanity.com/paper/2405.05145/) (2024)
- [Trustworthy Classification through Rank-Based Conformal Prediction Sets](https://mlsanity.com/paper/2407.04407/) (2024)
- [Conformal Linguistic Calibration: Trading-off between Factuality and Specificity](https://mlsanity.com/paper/2502.19110/) (2025)
- [Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers](https://mlsanity.com/paper/2208.11111/) (2022)
- [Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration](https://mlsanity.com/paper/2406.06818/) (2024)

## Abstract (arXiv)

We extend conformal prediction to control the expected value of any monotone loss function. The algorithm generalizes split conformal prediction together with its coverage guarantee. Like conformal prediction, the conformal risk control procedure is tight up to an O(1/n) factor. We also introduce extensions of the idea to distribution shift, quantile risk control, multiple and adversarial risk control, and expectations of U-statistics. Worked examples from computer vision and natural language processing demonstrate the usage of our algorithm to bound the false negative rate, graph distance, and token-level F1-score.