mlsanityData to 7 Oct 2026

Learning Optimal Conformal Classifiers

arXiv 2110.09192 ↗First posted 18 Oct 2021cs.LG cs.CV stat.MEcited by 108

Not on the map. Why?

Abstract (arXiv)

Modern deep learning based classifiers show very high accuracy on test data but this does not provide sufficient guarantees for safe deployment, especially in high-stake AI applications such as medical diagnosis. Usually, predictions are obtained without a reliable uncertainty estimate or a formal guarantee. Conformal prediction (CP) addresses these issues by using the classifier's predictions, e.g., its probability estimates, to predict confidence sets containing the true class with a user-specified probability. However, using CP as a separate processing step after training prevents the underlying model from adapting to the prediction of confidence sets. Thus, this paper explores strategies to differentiate through CP during training with the goal of training model with the conformal wrapper end-to-end. In our approach, conformal training (ConfTr), we specifically "simulate" conformalization on mini-batches during training. Compared to standard training, ConfTr reduces the average confidence set size (inefficiency) of state-of-the-art CP methods applied after training. Moreover, it allows to "shape" the confidence sets predicted at test time, which is difficult for standard CP. On experiments with several datasets, we show ConfTr can influence how inefficiency is distributed across classes, or guide the composition of confidence sets in terms of the included classes, while retaining the guarantees offered by CP.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 21 of its 47 references that are in the map. First posted on arXiv 18 Oct 2021.

Builds on

21 of 47 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  6. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  7. 2019Conformalized Quantile Regressioncited by 620
  8. 2020Classification with Valid and Adaptive Coveragecited by 390
  9. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  10. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  11. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  12. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
9 more
  1. 2019Predictive inference with the jackknife+cited by 206
  2. 2020Fast Differentiable Sorting and Rankingcited by 196
  3. 2020Learning to Complement Humanscited by 123
  4. 2019A deep learning system for differential diagnosis of skin diseasescited by 118
  5. 2019Differentiable Ranks and Sorting using Optimal Transportcited by 113
  6. 2013A Conformal Prediction Approach to Explore Functional Datacited by 85
  7. 2021Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditionscited by 68
  8. 2018Cautious Deep Learningcited by 35
  9. 2021Optimized conformal classification using gradient descent approximationcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 18; 2024: 31; 2025: 30; 2026: 20.

108 papers.

Sort:

By area:

  1. 2024Large language model validity via enhanced conformal prediction methodscited by 91
  2. 2023Uncertainty Quantification over Graph with Conformalized Graph Neural Networkscited by 79
  3. 2023Improved Online Conformal Prediction via Strongly Adaptive Online Learningcited by 76
  4. 2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 62
  5. 2023Conformal Prediction for Deep Classifier via Label Rankingcited by 52
  6. 2024Length Optimization in Conformal Predictioncited by 39
  7. 2022On the Utility of Prediction Sets in Human-AI Teamscited by 34
  8. 2022Predictive Inference with Feature Conformal Predictioncited by 33
  9. 2023Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagationcited by 33
  10. 2023Conformal Prediction Regions for Time Series using Linear Complementarity Programmingcited by 33
  11. 2023Conformal prediction under ambiguous ground truthcited by 31
  12. 2022Distribution Free Prediction Sets for Node Classificationcited by 28
38 more
  1. 2022Bayesian Optimization with Conformal Prediction Setscited by 27
  2. 2024Provably Robust Conformal Prediction with Improved Efficiencycited by 27
  3. 2022Conformal Off-Policy Prediction in Contextual Banditscited by 25
  4. 2024Boosted Conformal Prediction Intervalscited by 25
  5. 2025E-Values Expand the Scope of Conformal Predictioncited by 23
  6. 2024Conformalized Credal Set Predictorscited by 21
  7. 2024An Information Theoretic Perspective on Conformal Predictioncited by 21
  8. 2022Split Localized Conformal Predictioncited by 20
  9. 2024Trustworthy Classification through Rank-Based Conformal Prediction Setscited by 20
  10. 2023Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Controlcited by 19
  11. 2022Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Predictioncited by 17
  12. 2024Robust Yet Efficient Conformal Prediction Setscited by 16
  13. 2024Prune 'n Predict: Optimizing LLM Decision-making with Conformal Predictioncited by 15
  14. 2023Conformal inference is (almost) free for neural networks trained with early stoppingcited by 14
  15. 2024End-to-End Conformal Calibration for Optimization Under Uncertaintycited by 14
  16. 2025Wasserstein-regularized Conformal Prediction under General Distribution Shiftcited by 14
  17. 2025Conformal Prediction for Zero-Shot Modelscited by 14
  18. 2023On the Expected Size of Conformal Prediction Setscited by 13
  19. 2024On Temperature Scaling and Conformal Prediction of Deep Classifierscited by 13
  20. 2025Backward Conformal Predictioncited by 13
  21. 2024Does confidence calibration improve conformal prediction?cited by 12
  22. 2024Are foundation models for computer vision good conformal predictors?cited by 12
  23. 2024Normalizing Flows for Conformal Regressioncited by 11
  24. 2024Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimizationcited by 11
  25. 2025Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularizationcited by 11
  26. 2025Conformal Risk Training: End-to-End Optimization of Conformal Risk Controlcited by 11
  27. 2024Conformal Risk Minimization with Variance Reductioncited by 10
  28. 2024Game-Theoretic Defenses for Robust Conformal Prediction Against Adversarial Attacks in Medical Imagingcited by 10
  29. 2024Generalization and Informativeness of Conformal Predictioncited by 9
  30. 2024TorchCP: A Python Library for Conformal Predictioncited by 9
  31. 2024Conformal Inductive Graph Neural Networkscited by 9
  32. 2025Volume Optimality in Conformal Prediction with Structured Prediction Setscited by 9
  33. 2025Full Conformal Adaptation of Medical Vision-Language Modelscited by 9
  34. 2023Knowing When to Stop: Delay-Adaptive Spiking Neural Network Classifiers with Reliability Guaranteescited by 8
  35. 2023On training locally adaptive CPcited by 8
  36. 2024Cross-Validation Conformal Risk Controlcited by 8
  37. 2025Direct Prediction Set Minimization via Bilevel Conformal Classifier Trainingcited by 8
  38. 2023Empirically Validating Conformal Prediction on Modern Vision Architectures Under Distribution Shift and Long-tailed Datacited by 7

50 most cited of 108 All 108 (JSON)