mlsanityData to 7 Oct 2026

Conformalized Credal Set Predictors

arXiv 2402.10723 ↗First posted 16 Feb 2024stat.ML cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

Credal sets are sets of probability distributions that are considered as candidates for an imprecisely known ground-truth distribution. In machine learning, they have recently attracted attention as an appealing formalism for uncertainty representation, in particular due to their ability to represent both the aleatoric and epistemic uncertainty in a prediction. However, the design of methods for learning credal set predictors remains a challenging problem. In this paper, we make use of conformal prediction for this purpose. More specifically, we propose a method for predicting credal sets in the classification task, given training data labeled by probability distributions. Since our method inherits the coverage guarantees of conformal prediction, our conformal credal sets are guaranteed to be valid with high probability (without any assumptions on model or distribution). We demonstrate the applicability of our method to natural language inference, a highly ambiguous natural language task where it is common to obtain multiple annotations per example.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 32 of its 65 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

32 of 65 references.

References · 32

  1. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  2. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
  6. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  7. 2019Conformalized Quantile Regressioncited by 620
  8. 2020Classification with Valid and Adaptive Coveragecited by 390
  9. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  10. 2020Are we done with ImageNet?cited by 329
  11. 2019Abductive Commonsense Reasoningcited by 300
  12. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
20 more
  1. 2019Human uncertainty makes classification more robustcited by 231
  2. 2019Predictive inference with the jackknife+cited by 206
  3. 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 148
  4. 2021MultiVerS: Improving scientific claim verification with weak supervision and full-document contextcited by 140
  5. 2021Learning Optimal Conformal Classifierscited by 108
  6. 2021Conformal Prediction using Conditional Histogramscited by 77
  7. 2022Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisationcited by 65
  8. 2022Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimationcited by 43
  9. 2020Aleatoric and Epistemic Uncertainty with Random Forestscited by 42
  10. 2023Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?cited by 40
  11. 2023On Second-Order Scoring Rules for Epistemic Uncertainty Quantificationcited by 38
  12. 2016Crowdsourcing in Computer Visioncited by 31
  13. 2023Conformal prediction under ambiguous ground truthcited by 31
  14. 2022Conformal Prediction Sets with Limited False Positivescited by 28
  15. 2023Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreementcited by 22
  16. 2019A Crowdsourced Frame Disambiguation Corpus with Ambiguitycited by 16
  17. 2022Conformal Prediction Intervals for Markov Decision Process Trajectoriescited by 12
  18. 2020Validity, consonant plausibility measures, and conformal predictioncited by 9
  19. 2022Conformal Credal Self-Supervised Learningcited by 8
  20. 2023Conformal Prediction with Partially Labeled Datacited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 11.

21 papers.

Sort:

  1. 2024Conformalized Credal Regions for Classification with Ambiguous Ground Truthcited by 19
  2. 2024Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusioncited by 9
  3. 2025Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'cited by 8
  4. 2025Credal Prediction based on Relative Likelihoodcited by 8
  5. 2025Conformal Prediction Regions are Imprecise Highest Density Regionscited by 7
  6. 2025A calibration test for evaluating set-based epistemic uncertainty representationscited by 7
  7. 2026Quantifying Epistemic Predictive Uncertainty in Conformal Predictioncited by 4
  8. 2025Conformal Prediction with Corrupted Labels: Uncertain Imputation and Robust Re-weightingcited by 3
  9. 2025Optimal Conformal Prediction under Epistemic Uncertaintycited by 3
  10. 2025Distilling Calibration via Conformalized Credal Inferencecited by 2
  11. 2026Credal Large Language Models for Semantic Commitment under Uncertaintycited by 2
  12. 2024Mislabeled examples detection viewed as probing machine learning models: concepts, survey and extensive benchmarkcited by 1
9 more
  1. 2026Structured Credal Learningcited by 1
  2. 2026Empirical Bayes Conformal Prediction for Vision and Language Modelscited by 1
  3. 2026Structurally Separated Uncertainty in Supervised Latent Variable Modelscited by 0
  4. 2026CREDO: Epistemic-Aware Conformalized Credal Envelopes for Regressioncited by 0
  5. 2026Efficient Credal Prediction through Decalibrationcited by 0
  6. 2026Upper Entropy for 2-Monotone Lower Probabilitiescited by 0
  7. 2026ConformaDecompose: Explaining Uncertainty via Calibration Localizationcited by 0
  8. 2026On the QUEST for Uncertainty Quantification via Highest Density Regionscited by 0
  9. 2026SimplexUQ: An Evaluation Framework and Benchmark for Conformal Uncertainty on Simplex-Valued Predictionscited by 0