mlsanityData to 7 Oct 2026

Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling

arXiv 2401.08876 ↗First posted 16 Jan 2024cs.HC cs.CV cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

As deep neural networks are more commonly deployed in high-stakes domains, their black-box nature makes uncertainty quantification challenging. We investigate the presentation of conformal prediction sets--a distribution-free class of methods for generating prediction sets with specified coverage--to express uncertainty in AI-advised decision-making. Through a large online experiment, we compare the utility of conformal prediction sets to displays of Top-1 and Top-k predictions for AI-advised image labeling. In a pre-registered analysis, we find that the utility of prediction sets for accuracy varies with the difficulty of the task: while they result in accuracy on par with or less than Top-1 and Top-k displays for easy images, prediction sets offer some advantage in assisting humans in labeling out-of-distribution (OOD) images in the setting that we studied, especially when the set size is small. Our results empirically pinpoint practical challenges of conformal prediction sets and provide implications on how to incorporate them for real-world decision-making.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 27 of its 68 references that are in the map. First posted on arXiv 16 Jan 2024.

Builds on

27 of 68 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2016Wide Residual Networkscited by 3,057
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  8. 2016Concrete Problems in AI Safetycited by 1,582
  9. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  10. 2019Measuring Calibration in Deep Learningcited by 443
  11. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  12. 2020Classification with Valid and Adaptive Coveragecited by 390
15 more
  1. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  2. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  3. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  4. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  5. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  6. 2017Deep Learning for Medical Image Processing: Overview, Challenges and Futurecited by 104
  7. 2022Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderationcited by 66
  8. 2021Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Makingcited by 58
  9. 2020Bayesian Workflowcited by 47
  10. 2023Conformal prediction with local weights: randomization enables local guaranteescited by 47
  11. 2022On the Utility of Prediction Sets in Human-AI Teamscited by 34
  12. 2021Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiativecited by 28
  13. 2023Designing Decision Support Systems Using Counterfactual Prediction Setscited by 26
  14. 2021A survey of image labelling for computer vision applicationscited by 18
  15. 2021Visualizing Uncertainty in Probabilistic Graphs with Network Hypothetical Outcome Plots (NetHOPs)cited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 4.

12 papers.

  1. 2024Towards Human-AI Complementarity with Prediction Setscited by 25
  2. 2024Conformal Prediction Sets Can Cause Disparate Impactcited by 14
  3. 2024Controlling Counterfactual Harm in Decision Support Systems Based on Prediction Setscited by 9
  4. 2025Conformal Prediction and Human Decision Makingcited by 8
  5. 2025Conformal Prediction for Long-Tailed Classificationcited by 7
  6. 2025Human-AI Collaborative Uncertainty Quantificationcited by 6
  7. 2026Conformal Policy Controlcited by 5
  8. 2025Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Makingcited by 4
  9. 2025Test-time augmentation improves efficiency in conformal predictioncited by 3
  10. 2026A Framework for Measuring Appropriate Reliance on Set-Valued AI Advicecited by 1
  11. 2026Questioning the Coverage-Length Metric in Conformal Prediction: When Shorter Intervals Are Not Bettercited by 0
  12. 2026Toward Human-AI Complementarity Across Diverse Taskscited by 0