# Enhancing the Reliability of Medical AI through Expert-guided Uncertainty Modeling

arXiv: https://arxiv.org/abs/2604.01898 · HTML: https://mlsanity.com/paper/2604.01898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 56 references that are in the map.
- First posted on arXiv 2 Apr 2026.

## Builds on (most cited first, 25 of 25)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019)
- [A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1910.02923/) (2019)
- [Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?](https://mlsanity.com/paper/2209.03302/) (2022)
- [Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration](https://mlsanity.com/paper/2102.12182/) (2021)
- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)
- [Estimating Expected Calibration Errors](https://mlsanity.com/paper/2109.03480/) (2021)
- [Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks](https://mlsanity.com/paper/2208.00461/) (2022)
- [Label-wise Aleatoric and Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2406.02354/) (2024)
- [Hallucination Detection in LLMs with Topological Divergence on Attention Graphs](https://mlsanity.com/paper/2504.10063/) (2025)
- [Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI](https://mlsanity.com/paper/2409.09467/) (2024)
- [Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models](https://mlsanity.com/paper/2312.15297/) (2023)
- [Evidential Uncertainty Quantification: A Variance-Based Perspective](https://mlsanity.com/paper/2311.11367/) (2023)
- [Expert-Augmented Machine Learning](https://mlsanity.com/paper/1903.09731/) (2019)
- [A Survey on Image Quality Assessment: Insights, Analysis, and Future Outlook](https://mlsanity.com/paper/2502.08540/) (2025)
- [KINN: Incorporating Expert Knowledge in Neural Networks](https://mlsanity.com/paper/1902.05653/) (2019)

## Abstract (arXiv)

Artificial intelligence (AI) systems accelerate medical workflows and improve diagnostic accuracy in healthcare, serving as second-opinion systems. However, the unpredictability of AI errors poses a significant challenge, particularly in healthcare contexts, where mistakes can have severe consequences. A widely adopted safeguard is to pair predictions with uncertainty estimation, enabling human experts to focus on high-risk cases while streamlining routine verification. Current uncertainty estimation methods, however, remain limited, particularly in quantifying aleatoric uncertainty, which arises from data ambiguity and noise. To address this, we propose a novel approach that leverages disagreement in expert responses to generate targets for training machine learning models. These targets are used in conjunction with standard data labels to estimate two components of uncertainty separately, as given by the law of total variance, via a two-ensemble approach, as well as its lightweight variant. We validate our method on binary image classification, binary and multi-class image segmentation, and multiple-choice question answering. Our experiments demonstrate that incorporating expert knowledge can enhance uncertainty estimation quality by 9% to 50% depending on the task, making this source of information invaluable for the construction of risk-aware AI systems in healthcare applications.