# Automatic quality control framework for more reliable integration of machine learning-based image segmentation into medical workflows

arXiv: https://arxiv.org/abs/2112.03277 · HTML: https://mlsanity.com/paper/2112.03277/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 6 Dec 2021.

## Builds on (most cited first, 24 of 24)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks](https://mlsanity.com/paper/1807.07356/) (2018)
- [Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation](https://mlsanity.com/paper/1808.01200/) (2018)
- [Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge](https://mlsanity.com/paper/1904.00682/) (2019)
- [CNN-based Segmentation of Medical Imaging Data](https://mlsanity.com/paper/1701.03056/) (2017)
- [Leveraging Uncertainty Estimates for Predicting Segmentation Quality](https://mlsanity.com/paper/1807.00502/) (2018)
- [Supervised Uncertainty Quantification for Segmentation with Multiple Annotations](https://mlsanity.com/paper/1907.01949/) (2019)
- [Bayesian QuickNAT: Model Uncertainty in Deep Whole-Brain Segmentation for Structure-wise Quality Control](https://mlsanity.com/paper/1811.09800/) (2018)
- [Uncertainty Estimation Using a Single Deep Deterministic Neural Network](https://mlsanity.com/paper/2003.02037/) (2020)
- [Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth](https://mlsanity.com/paper/1702.03407/) (2017)
- [Towards safe deep learning: accurately quantifying biomarker uncertainty in neural network predictions](https://mlsanity.com/paper/1806.08640/) (2018)
- [Fully Convolutional Network Ensembles for White Matter Hyperintensities Segmentation in MR Images](https://mlsanity.com/paper/1802.05203/) (2018)
- [Towards increased trustworthiness of deep learning segmentation methods on cardiac MRI](https://mlsanity.com/paper/1809.10430/) (2018)
- [Challenges for machine learning in clinical translation of big data imaging studies](https://mlsanity.com/paper/2107.05630/) (2021)
- [Uncertainty-driven Sanity Check: Application to Postoperative Brain Tumor Cavity Segmentation](https://mlsanity.com/paper/1806.03106/) (2018)
- [An Alarm System For Segmentation Algorithm Based On Shape Model](https://mlsanity.com/paper/1903.10645/) (2019)
- [Accuracy, Uncertainty, and Adaptability of Automatic Myocardial ASL Segmentation using Deep CNN](https://mlsanity.com/paper/1812.03974/) (2018)
- [Prostate Segmentation from 3D MRI Using a Two-Stage Model and Variable-Input Based Uncertainty Measure](https://mlsanity.com/paper/1903.02500/) (2019)
- [A Fine-Grain Error Map Prediction and Segmentation Quality Assessment Framework for Whole-Heart Segmentation](https://mlsanity.com/paper/1907.12244/) (2019)
- [A deep learning-based pipeline for error detection and quality control of brain MRI segmentation results](https://mlsanity.com/paper/2005.13987/) (2020)
- [Collaborative Boundary-aware Context Encoding Networks for Error Map Prediction](https://mlsanity.com/paper/2006.14345/) (2020)
- [Robust Image Segmentation Quality Assessment](https://mlsanity.com/paper/1903.08773/) (2019)

## Built on by (most cited first, 1 of 1)

- [Conformal Performance Range Prediction for Segmentation Output Quality Control](https://mlsanity.com/paper/2407.13307/) (2024)

## Abstract (arXiv)

Machine learning algorithms underpin modern diagnostic-aiding software, which has proved valuable in clinical practice, particularly in radiology. However, inaccuracies, mainly due to the limited availability of clinical samples for training these algorithms, hamper their wider applicability, acceptance, and recognition amongst clinicians. We present an analysis of state-of-the-art automatic quality control (QC) approaches that can be implemented within these algorithms to estimate the certainty of their outputs. We validated the most promising approaches on a brain image segmentation task identifying white matter hyperintensities (WMH) in magnetic resonance imaging data. WMH are a correlate of small vessel disease common in mid-to-late adulthood and are particularly challenging to segment due to their varied size, and distributional patterns. Our results show that the aggregation of uncertainty and Dice prediction were most effective in failure detection for this task. Both methods independently improved mean Dice from 0.82 to 0.84. Our work reveals how QC methods can help to detect failed segmentation cases and therefore make automatic segmentation more reliable and suitable for clinical practice.