# Understanding implementation pitfalls of distance-based metrics for image segmentation

arXiv: https://arxiv.org/abs/2410.02630 · HTML: https://mlsanity.com/paper/2410.02630/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 9 of 9)

- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021)
- [Why rankings of biomedical image analysis competitions should be interpreted with care](https://mlsanity.com/paper/1806.02051/) (2018)
- [Towards a Guideline for Evaluation Metrics in Medical Image Segmentation](https://mlsanity.com/paper/2202.05273/) (2022)
- [Understanding metric-related pitfalls in image analysis validation](https://mlsanity.com/paper/2302.01790/) (2023)
- [pymia: A Python package for data handling and evaluation in deep learning-based medical image analysis](https://mlsanity.com/paper/2010.03639/) (2020)
- [Seg-metrics: a Python package to compute segmentation metrics](https://mlsanity.com/paper/2403.07884/) (2024)

## Built on by (most cited first, 3 of 3)

- [TransUNet-GradCAM: A Hybrid Transformer-U-Net with Self-Attention and Explainable Visualizations for Foot Ulcer Segmentation](https://mlsanity.com/paper/2508.03758/) (2025)
- [Knowledge-Guided Brain Tumor Segmentation via Synchronized Visual-Semantic-Topological Prior Fusion](https://mlsanity.com/paper/2507.08574/) (2025)
- [MeshMetrics: A Precise Implementation of Distance-Based Image Segmentation Metrics](https://mlsanity.com/paper/2509.05670/) (2025)

## Abstract (arXiv)

Distance-based metrics, such as the Hausdorff distance (HD), are widely used to validate segmentation performance in (bio)medical imaging. However, their implementation is complex, and critical differences across open-source tools remain largely unrecognized by the community. These discrepancies undermine benchmarking efforts, introduce bias in biomarker calculations, and potentially distort medical device development and clinical commissioning. In this study, we systematically dissect 11 open-source tools that implement distance-based metric computation by performing both a conceptual analysis of their computational steps and an empirical analysis on representative two- and three-dimensional image datasets. Alarmingly, we observed deviations in HD exceeding 100 mm and identified multiple statistically significant differences between tools - demonstrating that statistically significant improvements on the same set of segmentations can be achieved simply by selecting a particular implementation. These findings cast doubts on the validity of prior comparisons of results across studies without accounting for the differences in metric implementations. To address this, we provide practical recommendations for tool selection; additionally, our conceptual analysis informs about the future evolution of implementing open-source tools.