mlsanityData to 7 Oct 2026

Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets

arXiv 2207.02238 ↗First posted 5 Jul 2022cs.LG eess.IVcited by 38

Not on the map. Why?

Abstract (arXiv)

The regulatory approval and broad clinical deployment of medical AI have been hampered by the perception that deep learning models fail in unpredictable and possibly catastrophic ways. A lack of statistically rigorous uncertainty quantification is a significant factor undermining trust in AI results. Recent developments in distribution-free uncertainty quantification present practical solutions for these issues by providing reliability guarantees for black-box models on arbitrary data distributions as formally valid finite-sample prediction intervals. Our work applies these new uncertainty quantification methods -- specifically conformal prediction -- to a deep-learning model for grading the severity of spinal stenosis in lumbar spine MRI. We demonstrate a technique for forming ordinal prediction sets that are guaranteed to contain the correct stenosis severity within a user-defined probability (confidence interval). On a dataset of 409 MRI exams processed by the deep-learning model, the conformal method provides tight coverage with small prediction set sizes. Furthermore, we explore the potential clinical applicability of flagging cases with high uncertainty predictions (large prediction sets) by quantifying an increase in the prevalence of significant imaging abnormalities (e.g. motion artifacts, metallic artifacts, and tumors) that could degrade confidence in predictive performance when compared to a random sample of cases.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 5 Jul 2022.

Builds on

20 of 33 references.

References · 20

  1. 2019Natural Adversarial Examplescited by 1,401
  2. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  3. 2019Conformalized Quantile Regressioncited by 620
  4. 2020Classification with Valid and Adaptive Coveragecited by 390
  5. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  6. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  7. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  8. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  9. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
  10. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
  11. 2022Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imagingcited by 107
  12. 2019Nested conformal prediction and quantile out-of-bag ensemble methodscited by 88
8 more
  1. 2013A Conformal Prediction Approach to Explore Functional Datacited by 85
  2. 2021Conformal Prediction using Conditional Histogramscited by 77
  3. 2021Fair Conformal Predictors for Applications in Medical Imagingcited by 74
  4. 2019Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Controlcited by 60
  5. 2022Conformal Prediction Under Feedback Covariate Shift for Biomolecular Designcited by 53
  6. 2021Private Prediction Setscited by 16
  7. 2022Three Applications of Conformal Prediction for Rating Breast Density in Mammographycited by 6
  8. 2018DeepSPINE: Automated Lumbar Vertebral Segmentation, Disc-level Designation, and Spinal Stenosis Grading Using Deep Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 14; 2025: 13; 2026: 6.

38 papers.

Sort:

By area:

  1. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 131
  2. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 62
  3. 2023Federated Conformal Predictors for Distributed Uncertainty Quantificationcited by 46
  4. 2025Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Predictioncited by 26
  5. 2024Conformal Prediction: A Data Perspectivecited by 25
  6. 2024An Information Theoretic Perspective on Conformal Predictioncited by 21
  7. 2024Conformal Prediction via Regression-as-Classificationcited by 18
  8. 2024Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibrationcited by 18
  9. 2024A Conformal Prediction Score that is Robust to Label Noisecited by 9
  10. 2024Conformal Risk Control for Ordinal Classificationcited by 8
  11. 2024Task-Driven Uncertainty Quantification in Inverse Problems via Conformal Predictioncited by 8
  12. 2025Direct Prediction Set Minimization via Bilevel Conformal Classifier Trainingcited by 8
26 more
  1. 2025Trustworthy Few-Shot Transfer of Medical VLMs through Split Conformal Predictioncited by 8
  2. 2023Empirically Validating Conformal Prediction on Modern Vision Architectures Under Distribution Shift and Long-tailed Datacited by 7
  3. 2024Quantile Learn-Then-Test: Quantile-Based Risk Control for Hyperparameter Optimizationcited by 6
  4. 2025Pitfalls of Conformal Predictions for Medical Image Classificationcited by 6
  5. 2025WQLCP: Weighted Adaptive Conformal Prediction for Robust Uncertainty Quantification Under Distribution Shiftscited by 5
  6. 2025Conformal Prediction of Classifiers with Many Classes based on Noisy Labelscited by 4
  7. 2024Conformalized Selective Regressioncited by 3
  8. 2024Reliable Multi-View Learning with Conformal Prediction for Aortic Stenosis Classification in Echocardiographycited by 3
  9. 2023RR-CP: Reliable-Region-Based Conformal Prediction for Trustworthy Medical Image Classificationcited by 2
  10. 2024Distribution-free Conformal Prediction for Ordinal Classificationcited by 2
  11. 2024Conformal Depression Predictioncited by 2
  12. 2025Adaptive Conformal Guidance for Learning under Uncertaintycited by 2
  13. 2025Privacy-Preserving Conformal Prediction Under Local Differential Privacycited by 2
  14. 2024A conformalized learning of a prediction set with applications to medical imaging classificationcited by 1
  15. 2025Efficient Conformal Prediction for Regression Models under Label Noisecited by 1
  16. 2025Provably Minimum-Length Conformal Prediction Sets for Ordinal Classificationcited by 1
  17. 2026Reliable Conformal Prediction for Ordinal Classification Using the Ranked Probability Scorecited by 1
  18. 2024Conformalized High-Density Quantile Regression via Dynamic Prototypes-based Probability Density Estimationcited by 0
  19. 2025Risk-Sensitive Conformal Prediction for Catheter Placement Detection in Chest X-rayscited by 0
  20. 2025Conformal Prediction and Trustworthy AIcited by 0
  21. 2025Cost-Sensitive Conformal Training with Provably Controllable Learning Boundscited by 0
  22. 2026Trustworthy Endoscopic Super-Resolutioncited by 0
  23. 2026Beyond Uncertainty: Generalizable Failure Monitoring for Surgical Segmentation under Acquisition Degradationcited by 0
  24. 2026Uncertainty of Vision Medical Foundation Modelscited by 0
  25. 2026Robust Conformal Consensus: Multi-Agent LLM-as-a-Judge Interval Evaluation with Conformal Predictioncited by 0
  26. 2026CHOIR: heterogeneity-aware conformal prediction for crash injury severity across driver safety stratacited by 0