# Towards a Guideline for Evaluation Metrics in Medical Image Segmentation

arXiv: https://arxiv.org/abs/2202.05273 · HTML: https://mlsanity.com/paper/2202.05273/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 55 references that are in the map.
- First posted on arXiv 10 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1605.01397/) (2016)
- [A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation](https://mlsanity.com/paper/1810.07842/) (2018)
- [Deep Semantic Segmentation of Natural and Medical Images: A Review](https://mlsanity.com/paper/1910.07655/) (2019)
- [Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks](https://mlsanity.com/paper/1904.10030/) (2019)
- [Going Deep in Medical Image Analysis: Concepts, Methods, Challenges and Future Directions](https://mlsanity.com/paper/1902.05655/) (2019)
- [MIScnn: A Framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning](https://mlsanity.com/paper/1910.09308/) (2019)
- [Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection](https://mlsanity.com/paper/2101.08418/) (2021)
- [MISeval: a Metric Library for Medical Image Segmentation Evaluation](https://mlsanity.com/paper/2201.09395/) (2022)
- [An Analysis on Ensemble Learning optimized Medical Image Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2201.11440/) (2022)
- [Multi-Disease Detection in Retinal Imaging based on Ensembling Heterogeneous Deep Learning Models](https://mlsanity.com/paper/2103.14660/) (2021)

## Built on by (most cited first, 30 of 60)

- [An objective comparison of methods for augmented reality in laparoscopic liver resection by preoperative-to-intraoperative image fusion](https://mlsanity.com/paper/2401.15753/) (2024)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [Comparative validation of surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation in endoscopy: Results of the PhaKIR 2024 challenge](https://mlsanity.com/paper/2507.16559/) (2025)
- [Improving Knot Prediction in Wood Logs with Longitudinal Feature Propagation](https://mlsanity.com/paper/2308.11291/) (2023)
- [Seg-metrics: a Python package to compute segmentation metrics](https://mlsanity.com/paper/2403.07884/) (2024)
- [ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality](https://mlsanity.com/paper/2503.04522/) (2025)
- [Demystifying the Effect of Receptive Field Size in U-Net Models for Medical Image Segmentation](https://mlsanity.com/paper/2406.16701/) (2024)
- [Understanding implementation pitfalls of distance-based metrics for image segmentation](https://mlsanity.com/paper/2410.02630/) (2024)
- [Deep learning approaches to surgical video segmentation and object detection: A Scoping Review](https://mlsanity.com/paper/2502.16459/) (2025)
- [MISm: A Medical Image Segmentation Metric for Evaluation of weak labeled Data](https://mlsanity.com/paper/2210.13642/) (2022)
- [BenchCloudVision: A Benchmark Analysis of Deep Learning Approaches for Cloud Detection and Segmentation in Remote Sensing Imagery](https://mlsanity.com/paper/2402.13918/) (2024)
- [Semi-Supervised Medical Image Segmentation via Dual Networks](https://mlsanity.com/paper/2505.17690/) (2025)
- [Metric Design != Metric Behavior: Improving Metric Selection for the Unbiased Evaluation of Dimensionality Reduction](https://mlsanity.com/paper/2507.02225/) (2025)
- [Absolute Evaluation Measures for Machine Learning: A Survey](https://mlsanity.com/paper/2507.03392/) (2025)
- [Can Diffusion Models Bridge the Domain Gap in Cardiac MR Imaging?](https://mlsanity.com/paper/2508.06327/) (2025)
- [Towards the Automatic Segmentation, Modeling and Meshing of the Aortic Vessel Tree from Multicenter Acquisitions: An Overview of the SEG.A. 2023 Segmentation of the Aorta Challenge](https://mlsanity.com/paper/2510.24009/) (2025)
- [Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis](https://mlsanity.com/paper/2512.01534/) (2025)
- [Auditing Significance, Metric Choice, and Demographic Fairness in Medical AI Challenges](https://mlsanity.com/paper/2512.19091/) (2025)
- [Robust Tumor Segmentation with Hyperspectral Imaging and Graph Neural Networks](https://mlsanity.com/paper/2311.11782/) (2023)
- [Exploring the Role of Convolutional Neural Networks (CNN) in Dental Radiography Segmentation: A Comprehensive Systematic Literature Review](https://mlsanity.com/paper/2401.09190/) (2024)
- [CT evaluation of 2D and 3D holistic deep learning methods for the volumetric segmentation of airway lesions](https://mlsanity.com/paper/2403.08042/) (2024)
- [DualSwinUnet++: An Enhanced Swin-Unet Architecture With Dual Decoders For PTMC Segmentation](https://mlsanity.com/paper/2410.18239/) (2024)
- [Continuous and complete liver vessel segmentation with graph-attention guided diffusion](https://mlsanity.com/paper/2411.00617/) (2024)
- [From Cold Start to Active Learning: Embedding-Based Scan Selection for Medical Image Segmentation](https://mlsanity.com/paper/2601.18532/) (2026)
- [Evaluation of Data Augmentation and Loss Functions in Semantic Image Segmentation for Drilling Tool Wear Detection](https://mlsanity.com/paper/2302.05262/) (2023)
- [AI-based Automatic Segmentation of Prostate on Multi-modality Images: A Review](https://mlsanity.com/paper/2407.06612/) (2024)
- [DiffRenderGAN: Addressing Training Data Scarcity in Deep Segmentation Networks for Quantitative Nanomaterial Analysis through Differentiable Rendering and Generative Modelling](https://mlsanity.com/paper/2502.09477/) (2025)
- [An Efficient Plugin Method for Metric Optimization of Black-Box Models](https://mlsanity.com/paper/2503.02119/) (2025)
- [Manifold Learning for Hyperspectral Images](https://mlsanity.com/paper/2503.15016/) (2025)
- [Enhancing Orthopox Image Classification Using Hybrid Machine Learning and Deep Learning Models](https://mlsanity.com/paper/2506.06007/) (2025)

## Abstract (arXiv)

In the last decade, research on artificial intelligence has seen rapid growth with deep learning models, especially in the field of medical image segmentation. Various studies demonstrated that these models have powerful prediction capabilities and achieved similar results as clinicians. However, recent studies revealed that the evaluation in image segmentation studies lacks reliable model performance assessment and showed statistical bias by incorrect metric implementation or usage. Thus, this work provides an overview and interpretation guide on the following metrics for medical image segmentation evaluation in binary as well as multi-class problems: Dice similarity coefficient, Jaccard, Sensitivity, Specificity, Rand index, ROC curves, Cohen's Kappa, and Hausdorff distance. As a summary, we propose a guideline for standardized medical image segmentation evaluation to improve evaluation quality, reproducibility, and comparability in the research field.