# Quantifying Deep Learning Model Uncertainty in Conformal Prediction

arXiv: https://arxiv.org/abs/2306.00876 · HTML: https://mlsanity.com/paper/2306.00876/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach](https://mlsanity.com/paper/1802.07167/) (2018)
- [On Reject and Refine Options in Multicategory Classification](https://mlsanity.com/paper/1701.02265/) (2017)

## Built on by (most cited first, 13 of 13)

- [Conformalized Multimodal Uncertainty Regression and Reasoning](https://mlsanity.com/paper/2309.11018/) (2023)
- [Evidential Uncertainty Estimation for Multi-Modal Trajectory Prediction](https://mlsanity.com/paper/2503.05274/) (2025)
- [Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry](https://mlsanity.com/paper/2602.01667/) (2026)
- [Evidential Uncertainty Sets in Deep Classifiers Using Conformal Prediction](https://mlsanity.com/paper/2406.10787/) (2024)
- [Adaptive Conformal Guidance for Learning under Uncertainty](https://mlsanity.com/paper/2502.16736/) (2025)
- [Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction](https://mlsanity.com/paper/2606.01850/) (2026)
- [Learning Enhanced Structural Representations with Block-Based Uncertainties for Ocean Floor Mapping](https://mlsanity.com/paper/2504.14372/) (2025)
- [neuralGAM: An R Package for Fitting Generalized Additive Neural Networks](https://mlsanity.com/paper/2505.08610/) (2025)
- [CARE: Confidence-aware Ratio Estimation for Medical Biomarkers](https://mlsanity.com/paper/2505.19585/) (2025)
- [GNN's Uncertainty Quantification using Self-Distillation](https://mlsanity.com/paper/2506.20046/) (2025)
- [INQUIRE-Search: Interactive Discovery in Large-Scale Biodiversity Databases](https://mlsanity.com/paper/2511.15656/) (2025)
- [Enhanced Data-Driven Product Development via Gradient Based Optimization and Conformalized Monte Carlo Dropout Uncertainty Estimation](https://mlsanity.com/paper/2601.00932/) (2026)
- [A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification](https://mlsanity.com/paper/2608.29789/) (2026)

## Abstract (arXiv)

Precise estimation of predictive uncertainty in deep neural networks is a critical requirement for reliable decision-making in machine learning and statistical modeling, particularly in the context of medical AI. Conformal Prediction (CP) has emerged as a promising framework for representing the model uncertainty by providing well-calibrated confidence levels for individual predictions. However, the quantification of model uncertainty in conformal prediction remains an active research area, yet to be fully addressed. In this paper, we explore state-of-the-art CP methodologies and their theoretical foundations. We propose a probabilistic approach in quantifying the model uncertainty derived from the produced prediction sets in conformal prediction and provide certified boundaries for the computed uncertainty. By doing so, we allow model uncertainty measured by CP to be compared by other uncertainty quantification methods such as Bayesian (e.g., MC-Dropout and DeepEnsemble) and Evidential approaches.