# Conformalized Credal Set Predictors

arXiv: https://arxiv.org/abs/2402.10723 · HTML: https://mlsanity.com/paper/2402.10723/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 65 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 30 of 32)

- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Abductive Commonsense Reasoning](https://mlsanity.com/paper/1908.05739/) (2019)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Human uncertainty makes classification more robust](https://mlsanity.com/paper/1908.07086/) (2019)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [What Can We Learn from Collective Human Opinions on Natural Language Inference Data?](https://mlsanity.com/paper/2010.03532/) (2020)
- [MultiVerS: Improving scientific claim verification with weak supervision and full-document context](https://mlsanity.com/paper/2112.01640/) (2021)
- [Learning Optimal Conformal Classifiers](https://mlsanity.com/paper/2110.09192/) (2021)
- [Conformal Prediction using Conditional Histograms](https://mlsanity.com/paper/2105.08747/) (2021)
- [Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation](https://mlsanity.com/paper/2203.06102/) (2022)
- [Aleatoric and Epistemic Uncertainty with Random Forests](https://mlsanity.com/paper/2001.00893/) (2020)
- [Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimation](https://mlsanity.com/paper/2207.06214/) (2022)
- [Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?](https://mlsanity.com/paper/2306.09586/) (2023)
- [On Second-Order Scoring Rules for Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2301.12736/) (2023)
- [Crowdsourcing in Computer Vision](https://mlsanity.com/paper/1611.02145/) (2016)
- [Conformal prediction under ambiguous ground truth](https://mlsanity.com/paper/2307.09302/) (2023)
- [Conformal Prediction Sets with Limited False Positives](https://mlsanity.com/paper/2202.07650/) (2022)
- [Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement](https://mlsanity.com/paper/2301.10684/) (2023)
- [A Crowdsourced Frame Disambiguation Corpus with Ambiguity](https://mlsanity.com/paper/1904.06101/) (2019)
- [Conformal Prediction Intervals for Markov Decision Process Trajectories](https://mlsanity.com/paper/2206.04860/) (2022)
- [Validity, consonant plausibility measures, and conformal prediction](https://mlsanity.com/paper/2001.09225/) (2020)

## Built on by (most cited first, 21 of 21)

- [Conformalized Credal Regions for Classification with Ambiguous Ground Truth](https://mlsanity.com/paper/2411.04852/) (2024)
- [Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusion](https://mlsanity.com/paper/2410.12592/) (2024)
- [Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'](https://mlsanity.com/paper/2505.04950/) (2025)
- [Credal Prediction based on Relative Likelihood](https://mlsanity.com/paper/2505.22332/) (2025)
- [Conformal Prediction Regions are Imprecise Highest Density Regions](https://mlsanity.com/paper/2502.06331/) (2025)
- [A calibration test for evaluating set-based epistemic uncertainty representations](https://mlsanity.com/paper/2502.16299/) (2025)
- [Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry](https://mlsanity.com/paper/2602.01667/) (2026)
- [Distilling Calibration via Conformalized Credal Inference](https://mlsanity.com/paper/2501.06066/) (2025)
- [Conformal Prediction with Corrupted Labels: Uncertain Imputation and Robust Re-weighting](https://mlsanity.com/paper/2505.04733/) (2025)
- [Optimal Conformal Prediction under Epistemic Uncertainty](https://mlsanity.com/paper/2505.19033/) (2025)
- [Credal Large Language Models for Semantic Commitment under Uncertainty](https://mlsanity.com/paper/2608.23244/) (2026)
- [Mislabeled examples detection viewed as probing machine learning models: concepts, survey and extensive benchmark](https://mlsanity.com/paper/2410.15772/) (2024)
- [Efficient Credal Prediction through Decalibration](https://mlsanity.com/paper/2603.08495/) (2026)
- [Structured Credal Learning](https://mlsanity.com/paper/2603.14070/) (2026)
- [Empirical Bayes Conformal Prediction for Vision and Language Models](https://mlsanity.com/paper/2605.23189/) (2026)
- [Structurally Separated Uncertainty in Supervised Latent Variable Models](https://mlsanity.com/paper/2602.11219/) (2026)
- [CREDO: Epistemic-Aware Conformalized Credal Envelopes for Regression](https://mlsanity.com/paper/2603.06826/) (2026)
- [Upper Entropy for 2-Monotone Lower Probabilities](https://mlsanity.com/paper/2603.23558/) (2026)
- [ConformaDecompose: Explaining Uncertainty via Calibration Localization](https://mlsanity.com/paper/2604.27149/) (2026)
- [On the QUEST for Uncertainty Quantification via Highest Density Regions](https://mlsanity.com/paper/2606.19569/) (2026)
- [SimplexUQ: An Evaluation Framework and Benchmark for Conformal Uncertainty on Simplex-Valued Predictions](https://mlsanity.com/paper/2610.00523/) (2026)

## Abstract (arXiv)

Credal sets are sets of probability distributions that are considered as candidates for an imprecisely known ground-truth distribution. In machine learning, they have recently attracted attention as an appealing formalism for uncertainty representation, in particular due to their ability to represent both the aleatoric and epistemic uncertainty in a prediction. However, the design of methods for learning credal set predictors remains a challenging problem. In this paper, we make use of conformal prediction for this purpose. More specifically, we propose a method for predicting credal sets in the classification task, given training data labeled by probability distributions. Since our method inherits the coverage guarantees of conformal prediction, our conformal credal sets are guaranteed to be valid with high probability (without any assumptions on model or distribution). We demonstrate the applicability of our method to natural language inference, a highly ambiguous natural language task where it is common to obtain multiple annotations per example.