# Fair Conformal Predictors for Applications in Medical Imaging

arXiv: https://arxiv.org/abs/2109.04392 · HTML: https://mlsanity.com/paper/2109.04392/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 9 Sep 2021.

## Builds on (most cited first, 27 of 27)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset](https://mlsanity.com/paper/2104.09957/) (2021)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [CheXclusion: Fairness gaps in deep chest X-ray classifiers](https://mlsanity.com/paper/2003.00827/) (2020)
- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [Secure and Robust Machine Learning for Healthcare: A Survey](https://mlsanity.com/paper/2001.08103/) (2020)
- [Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes](https://mlsanity.com/paper/1904.09612/) (2019)
- [With Malice Towards None: Assessing Uncertainty via Equalized Coverage](https://mlsanity.com/paper/1908.05428/) (2019)
- [Debiasing Skin Lesion Datasets and Models? Not So Fast](https://mlsanity.com/paper/2004.11457/) (2020)
- [CheXplain: Enabling Physicians to Explore and UnderstandData-Driven, AI-Enabled Medical Imaging Analysis](https://mlsanity.com/paper/2001.05149/) (2020)
- [Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lens](https://mlsanity.com/paper/2102.00593/) (2021)
- [Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging](https://mlsanity.com/paper/2008.02766/) (2020)
- [Evaluating subgroup disparity using epistemic uncertainty in mammography](https://mlsanity.com/paper/2107.02716/) (2021)

## Built on by (most cited first, 30 of 74)

- [Conformal Prediction with Large Language Models for Multi-Choice Question Answering](https://mlsanity.com/paper/2305.18404/) (2023)
- [Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging](https://mlsanity.com/paper/2202.05265/) (2022)
- [Federated Conformal Predictors for Distributed Uncertainty Quantification](https://mlsanity.com/paper/2305.17564/) (2023)
- [Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets](https://mlsanity.com/paper/2207.02238/) (2022)
- [Beyond Confidence: Reliable Models Should Also Consider Atypicality](https://mlsanity.com/paper/2305.18262/) (2023)
- [Conformal Prediction: A Data Perspective](https://mlsanity.com/paper/2410.06494/) (2024)
- [Multivariate Conformal Prediction using Optimal Transport](https://mlsanity.com/paper/2502.03609/) (2025)
- [An Information Theoretic Perspective on Conformal Prediction](https://mlsanity.com/paper/2405.02140/) (2024)
- [Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration](https://mlsanity.com/paper/2406.06818/) (2024)
- [Distribution-Free Federated Learning with Conformal Predictions](https://mlsanity.com/paper/2110.07661/) (2021)
- [Robust Yet Efficient Conformal Prediction Sets](https://mlsanity.com/paper/2407.09165/) (2024)
- [Adaptive Learn-then-Test: Statistically Valid and Efficient Hyperparameter Selection](https://mlsanity.com/paper/2409.15844/) (2024)
- [Conformal Prediction Sets Can Cause Disparate Impact](https://mlsanity.com/paper/2410.01888/) (2024)
- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [On the Expected Size of Conformal Prediction Sets](https://mlsanity.com/paper/2306.07254/) (2023)
- [A Conformal Prediction Score that is Robust to Label Noise](https://mlsanity.com/paper/2405.02648/) (2024)
- [Utility-Directed Conformal Prediction: A Decision-Aware Framework for Actionable Uncertainty Quantification](https://mlsanity.com/paper/2410.01767/) (2024)
- [Conformal Prediction under Levy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbations](https://mlsanity.com/paper/2502.14105/) (2025)
- [Empirical Validation of Conformal Prediction for Trustworthy Skin Lesions Classification](https://mlsanity.com/paper/2312.07460/) (2023)
- [Conformal Prediction and Human Decision Making](https://mlsanity.com/paper/2503.11709/) (2025)
- [Conformalized Interval Arithmetic with Symmetric Calibration](https://mlsanity.com/paper/2408.10939/) (2024)
- [Direct Prediction Set Minimization via Bilevel Conformal Classifier Training](https://mlsanity.com/paper/2506.06599/) (2025)
- [Trustworthy Few-Shot Transfer of Medical VLMs through Split Conformal Prediction](https://mlsanity.com/paper/2506.17503/) (2025)
- [VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation](https://mlsanity.com/paper/2604.25235/) (2026)
- [Mirror Online Conformal Prediction with Intermittent Feedback](https://mlsanity.com/paper/2503.10345/) (2025)
- [A Generic Framework for Conformal Fairness](https://mlsanity.com/paper/2505.16115/) (2025)
- [Three Applications of Conformal Prediction for Rating Breast Density in Mammography](https://mlsanity.com/paper/2206.12008/) (2022)
- [Guaranteed Dynamic Scheduling of Ultra-Reliable Low-Latency Traffic via Conformal Prediction](https://mlsanity.com/paper/2302.07675/) (2023)
- [Uncertainty-Aware Online Extrinsic Calibration: A Conformal Prediction Approach](https://mlsanity.com/paper/2501.06878/) (2025)
- [Pitfalls of Conformal Predictions for Medical Image Classification](https://mlsanity.com/paper/2506.18162/) (2025)

## Abstract (arXiv)

Deep learning has the potential to automate many clinically useful tasks in medical imaging. However translation of deep learning into clinical practice has been hindered by issues such as lack of the transparency and interpretability in these "black box" algorithms compared to traditional statistical methods. Specifically, many clinical deep learning models lack rigorous and robust techniques for conveying certainty (or lack thereof) in their predictions -- ultimately limiting their appeal for extensive use in medical decision-making. Furthermore, numerous demonstrations of algorithmic bias have increased hesitancy towards deployment of deep learning for clinical applications. To this end, we explore how conformal predictions can complement existing deep learning approaches by providing an intuitive way of expressing uncertainty while facilitating greater transparency to clinical users. In this paper, we conduct field interviews with radiologists to assess possible use-cases for conformal predictors. Using insights gathered from these interviews, we devise two clinical use-cases and empirically evaluate several methods of conformal predictions on a dermatology photography dataset for skin lesion classification. We show how to modify conformal predictions to be more adaptive to subgroup differences in patient skin tones through equalized coverage. Finally, we compare conformal prediction against measures of epistemic uncertainty.