mlsanityData to 7 Oct 2026

Conformal Prediction Sets with Limited False Positives

arXiv 2202.07650 ↗First posted 15 Feb 2022cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

We develop a new approach to multi-label conformal prediction in which we aim to output a precise set of promising prediction candidates with a bounded number of incorrect answers. Standard conformal prediction provides the ability to adapt to model uncertainty by constructing a calibrated candidate set in place of a single prediction, with guarantees that the set contains the correct answer with high probability. In order to obey this coverage property, however, conformal sets can become inundated with noisy candidates -- which can render them unhelpful in practice. This is particularly relevant to practical applications where there is a limited budget, and the cost (monetary or otherwise) associated with false positives is non-negligible. We propose to trade coverage for a notion of precision by enforcing that the presence of incorrect candidates in the predicted conformal sets (i.e., the total number of false positives) is bounded according to a user-specified tolerance. Subject to this constraint, our algorithm then optimizes for a generalized notion of set coverage (i.e., the true positive rate) that allows for any number of true answers for a given query (including zero). We demonstrate the effectiveness of this approach across a number of classification tasks in natural language processing, computer vision, and computational chemistry.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 27 of its 59 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

27 of 59 references.

References · 27

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  5. 2017Deep Setscited by 1,394
  6. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  7. 2017Selective Classification for Deep Neural Networkscited by 799
  8. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  9. 2019Conformalized Quantile Regressioncited by 620
  10. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
  11. 2020Classification with Valid and Adaptive Coveragecited by 390
  12. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 373
15 more
  1. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  2. 2019Verified Uncertainty Calibrationcited by 307
  3. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  4. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  5. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  6. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
  7. 2019Evaluating model calibration in classificationcited by 165
  8. 2021Testing for Outliers with Conformal p-valuescited by 133
  9. 2019Nested conformal prediction and quantile out-of-bag ensemble methodscited by 88
  10. 2020Distribution-free binary classification: prediction sets, confidence intervals and calibrationcited by 79
  11. 2020Uncertainty Quantification Using Neural Networks for Molecular Property Predictioncited by 66
  12. 2021Few-shot Conformal Prediction with Auxiliary Taskscited by 53
  13. 2016Criteria of efficiency for conformal predictioncited by 36
  14. 2021Bayes-optimal prediction with frequentist coverage controlcited by 20
  15. 2015Large-scale probabilistic predictors with and without guarantees of validitycited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 6; 2025: 2; 2026: 8.

28 papers.

Sort:

By area:

  1. 2022Confident Adaptive Language Modelingcited by 297
  2. 2022Conformal Risk Controlcited by 290
  3. 2023Conformal Language Modelingcited by 163
  4. 2023Conformal Prediction for Deep Classifier via Label Rankingcited by 52
  5. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  6. 2023Conformal prediction under ambiguous ground truthcited by 31
  7. 2024Conformal Prediction: A Data Perspectivecited by 25
  8. 2024Non-Exchangeable Conformal Language Generation with Nearest Neighborscited by 21
  9. 2024Conformalized Credal Set Predictorscited by 21
  10. 2025Selective Conformal Risk Controlcited by 10
  11. 2024Conformal Inductive Graph Neural Networkscited by 9
  12. 2023Adaptive Conformal Regression with Jackknife+ Rescaled Scorescited by 8
16 more
  1. 2026PASC: Pipeline-Aware Conformal Prediction with Joint Coverage Guarantees for Multi-Stage NLP and LLM Pipelinescited by 7
  2. 2023Max-Rank: Efficient Multiple Testing for Conformal Predictioncited by 6
  3. 2023Conformalizing Machine Translation Evaluationcited by 5
  4. 2024Conformal Intent Classification and Clarification for Fast and Accurate Intent Recognitioncited by 4
  5. 2023Conformal Loss-Controlling Predictioncited by 3
  6. 2023Fast Online Value-Maximizing Prediction Sets with Conformal Cost Controlcited by 3
  7. 2023RR-CP: Reliable-Region-Based Conformal Prediction for Trustworthy Medical Image Classificationcited by 2
  8. 2026Conformal Risk Control under Non-Monotone Losses: Theory and Finite-Sample Guaranteescited by 2
  9. 2026Conformal Prediction Sets for Instance Segmentationcited by 1
  10. 2026Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Boundscited by 1
  11. 2023CONFIDERAI: a novel CONFormal Interpretable-by-Design score function for Explainable and Reliable Artificial Intelligencecited by 0
  12. 2025Controlling False Positives in Image Segmentation via Conformal Predictioncited by 0
  13. 2026Uncertainty Quantification for Named Entity Recognition via Full-Sequence and Subsequence Conformal Predictioncited by 0
  14. 2026Questioning the Coverage-Length Metric in Conformal Prediction: When Shorter Intervals Are Not Bettercited by 0
  15. 2026Softmax is not Enough (for Adaptive Conformal Classification)cited by 0
  16. 2026SimplexUQ: An Evaluation Framework and Benchmark for Conformal Uncertainty on Simplex-Valued Predictionscited by 0