# Uncertainty-guided annotation enhances segmentation with the human-in-the-loop

arXiv: https://arxiv.org/abs/2404.07208 · HTML: https://mlsanity.com/paper/2404.07208/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 5 of 5)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology](https://mlsanity.com/paper/1902.06543/) (2019)
- [Diminishing Uncertainty within the Training Pool: Active Learning for Medical Image Segmentation](https://mlsanity.com/paper/2101.02323/) (2021)
- [Uncertainty-Informed Deep Learning Models Enable High-Confidence Predictions for Digital Histopathology](https://mlsanity.com/paper/2204.04516/) (2022)
- [Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations](https://mlsanity.com/paper/2107.12357/) (2021)

## Built on by (most cited first, 4 of 4)

- [A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation](https://mlsanity.com/paper/2411.16370/) (2024)
- [Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation](https://mlsanity.com/paper/2605.18329/) (2026)
- [THIRDEYE: Cue-Aware Monocular Depth Estimation via Brain-Inspired Multi-Stage Fusion](https://mlsanity.com/paper/2506.20877/) (2025)
- [Bridging 3D Deep Learning and Curation for Analysis and High-Quality Segmentation in Practice](https://mlsanity.com/paper/2511.22236/) (2025)

## Abstract (arXiv)

Deep learning algorithms, often critiqued for their 'black box' nature, traditionally fall short in providing the necessary transparency for trusted clinical use. This challenge is particularly evident when such models are deployed in local hospitals, encountering out-of-domain distributions due to varying imaging techniques and patient-specific pathologies. Yet, this limitation offers a unique avenue for continual learning. The Uncertainty-Guided Annotation (UGA) framework introduces a human-in-the-loop approach, enabling AI to convey its uncertainties to clinicians, effectively acting as an automated quality control mechanism. UGA eases this interaction by quantifying uncertainty at the pixel level, thereby revealing the model's limitations and opening the door for clinician-guided corrections. We evaluated UGA on the Camelyon dataset for lymph node metastasis segmentation which revealed that UGA improved the Dice coefficient (DC), from 0.66 to 0.76 by adding 5 patches, and further to 0.84 with 10 patches. To foster broader application and community contribution, we have made our code accessible at