mlsanityData to 7 Oct 2026

Training Uncertainty-Aware Classifiers with Conformalized Deep Learning

arXiv 2205.05878 ↗First posted 12 May 2022stat.ML cs.LGcited by 62

Not on the map. Why?

Abstract (arXiv)

Deep neural networks are powerful tools to detect hidden patterns in data and leverage them to make predictions, but they are not designed to understand uncertainty and estimate reliable probabilities. In particular, they tend to be overconfident. We begin to address this problem in the context of multi-class classification by developing a novel training algorithm producing models with more dependable uncertainty estimates, without sacrificing predictive power. The idea is to mitigate overconfidence by minimizing a loss function, inspired by advances in conformal inference, that quantifies model uncertainty by carefully leveraging hold-out data. Experiments with synthetic and real data demonstrate this method can lead to smaller conformal prediction sets with higher conditional coverage, after exact calibration with hold-out data, compared to state-of-the-art alternatives.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 49 of its 99 references that are in the map. First posted on arXiv 12 May 2022.

Builds on

49 of 99 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  7. 2017Random Erasing Data Augmentationcited by 1,235
  8. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  9. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  10. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  11. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  12. 2019Conformalized Quantile Regressioncited by 620
37 more
  1. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  2. 2019Conformal Prediction Under Covariate Shiftcited by 511
  3. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  4. 2020Classification with Valid and Adaptive Coveragecited by 390
  5. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  6. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  7. 2022Conformal prediction beyond exchangeabilitycited by 348
  8. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  9. 2019Verified Uncertainty Calibrationcited by 307
  10. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
  11. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  12. 2019Predictive inference with the jackknife+cited by 206
  13. 2018To Trust Or Not To Trust A Classifiercited by 205
  14. 2020Fast Differentiable Sorting and Rankingcited by 196
  15. 2019Evaluating model calibration in classificationcited by 165
  16. 2021Testing for Outliers with Conformal p-valuescited by 133
  17. 2019Differentiable Ranks and Sorting using Optimal Transportcited by 113
  18. 2021Learning Optimal Conformal Classifierscited by 108
  19. 2022Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imagingcited by 107
  20. 2021Conformal Prediction using Conditional Histogramscited by 77
  21. 2019Deep Gamblers: Learning to Abstain with Portfolio Theorycited by 69
  22. 2020Positive-Congruent Training: Towards Regression-Free Model Updatescited by 55
  23. 2020Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detectioncited by 46
  24. 2021Selection and Aggregation of Conformal Prediction Setscited by 46
  25. 2019Prediction and outlier detection in classification problemscited by 44
  26. 2019With Malice Towards None: Assessing Uncertainty via Equalized Coveragecited by 43
  27. 2018Deep Knockoffscited by 40
  28. 2018Cautious Deep Learningcited by 35
  29. 2018Distribution-Free Prediction Sets for Two-Layer Hierarchical Modelscited by 34
  30. 2021Optimized conformal classification using gradient descent approximationcited by 27
  31. 2022Efficient and Differentiable Conformal Prediction with General Function Classescited by 25
  32. 2020Training conformal predictorscited by 20
  33. 2021Learning Prediction Intervals for Regression: Generalization and Calibrationcited by 16
  34. 2020Knowing what you know: valid and validated confidence sets in multiclass and multilabel predictioncited by 9
  35. 2020On Calibration of Scene-Text Recognition Modelscited by 9
  36. 2020Wisdom of the Ensemble: Improving Consistency of Deep Learning Modelscited by 7
  37. 2020PIVEN: A Deep Neural Network for Prediction Intervals with Specific Value Predictioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 11; 2024: 21; 2025: 17; 2026: 8.

62 papers.

Sort:

By area:

  1. 2024Language Models with Conformal Factuality Guaranteescited by 128
  2. 2023Conformal Prediction for Deep Classifier via Label Rankingcited by 52
  3. 2024Impact of Preference Noise on the Alignment Performance of Generative Language Modelscited by 41
  4. 2022Distribution Free Prediction Sets for Node Classificationcited by 28
  5. 2024Provably Robust Conformal Prediction with Improved Efficiencycited by 27
  6. 2024Conformal Prediction: A Data Perspectivecited by 25
  7. 2023Adaptive conformal classification with noisy labelscited by 24
  8. 2022Split Localized Conformal Predictioncited by 20
  9. 2024Uncertainty in Graph Neural Networks: A Surveycited by 20
  10. 2022Integrative conformal p-values for powerful out-of-distribution testing with labeled outlierscited by 19
  11. 2022Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Predictioncited by 17
  12. 2023Equal Opportunity of Coverage in Fair Regressioncited by 16
38 more
  1. 2023Conformal inference is (almost) free for neural networks trained with early stoppingcited by 14
  2. 2023A Data-Driven Measure of Relative Uncertainty for Misclassification Detectioncited by 14
  3. 2024End-to-End Conformal Calibration for Optimization Under Uncertaintycited by 14
  4. 2025Wasserstein-regularized Conformal Prediction under General Distribution Shiftcited by 14
  5. 2025Conformal Prediction for Zero-Shot Modelscited by 14
  6. 2023Lightweight, Uncertainty-Aware Conformalized Visual Odometrycited by 13
  7. 2024Does confidence calibration improve conformal prediction?cited by 12
  8. 2024Are foundation models for computer vision good conformal predictors?cited by 12
  9. 2024Normalizing Flows for Conformal Regressioncited by 11
  10. 2025Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularizationcited by 11
  11. 2025Conformal Risk Training: End-to-End Optimization of Conformal Risk Controlcited by 11
  12. 2024Conformal Risk Minimization with Variance Reductioncited by 10
  13. 2024TorchCP: A Python Library for Conformal Predictioncited by 9
  14. 2025Synthetic-Powered Predictive Inferencecited by 9
  15. 2025Full Conformal Adaptation of Medical Vision-Language Modelscited by 9
  16. 2023Knowing When to Stop: Delay-Adaptive Spiking Neural Network Classifiers with Reliability Guaranteescited by 8
  17. 2023On training locally adaptive CPcited by 8
  18. 2023SymmPI: Predictive Inference for Data with Group Symmetriescited by 8
  19. 2024Cross-Validation Conformal Risk Controlcited by 8
  20. 2024The Pitfalls and Promise of Conformal Inference Under Adversarial Attackscited by 8
  21. 2024Entropy Reweighted Conformal Classificationcited by 8
  22. 2025Direct Prediction Set Minimization via Bilevel Conformal Classifier Trainingcited by 8
  23. 2024C-Adapter: Adapting Deep Classifiers for Efficient Conformal Prediction Setscited by 7
  24. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
  25. 2023Conformalized Multimodal Uncertainty Regression and Reasoningcited by 6
  26. 2023PAC-Bayes Generalization Certificates for Learned Inductive Conformal Predictioncited by 6
  27. 2024Probabilistic Calibration by Design for Neural Network Regressioncited by 6
  28. 2024Error-Driven Uncertainty Aware Trainingcited by 3
  29. 2025Test-time augmentation improves efficiency in conformal predictioncited by 3
  30. 2025Uncertainty Estimation by Human Perception versus Neural Modelscited by 3
  31. 2024Two-Stage Surrogate Modeling for Data-Driven Design Optimization with Application to Composite Microstructure Generationcited by 2
  32. 2025CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignmentcited by 2
  33. 2025Conformal Prediction for Signal Temporal Logic Inferencecited by 2
  34. 2024A conformalized learning of a prediction set with applications to medical imaging classificationcited by 1
  35. 2025Advancing Image Classification with Discrete Diffusion Classification Modelingcited by 1
  36. 2022Calibrating AI Models for Wireless Communications via Conformal Predictioncited by 0
  37. 2024Adjusting Regression Models for Conditional Uncertainty Calibrationcited by 0
  38. 2024Conformal-in-the-Loop for Learning with Imbalanced Noisy Datacited by 0

50 most cited of 62 All 62 (JSON)