# In search of truth: Evaluating concordance of AI-based anatomy segmentation models

arXiv: https://arxiv.org/abs/2512.15921 · HTML: https://mlsanity.com/paper/2512.15921/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 65 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Builds on (most cited first, 13 of 13)

- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation](https://mlsanity.com/paper/2303.14444/) (2023)
- [Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth](https://mlsanity.com/paper/1702.03407/) (2017)
- [VIBESegmentator: Full Body MRI Segmentation for the NAKO and UK Biobank](https://mlsanity.com/paper/2406.00125/) (2024)
- [CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography](https://mlsanity.com/paper/2507.22953/) (2025)
- [MRISegmentator-Abdomen: A Fully Automated Multi-Organ and Structure Segmentation Tool for T1-weighted Abdominal MRI](https://mlsanity.com/paper/2405.05944/) (2024)
- [AI-Generated Annotations Dataset for Diverse Cancer Radiology Collections in NCI Image Data Commons](https://mlsanity.com/paper/2310.14897/) (2023)
- [Towards Ground-truth-free Evaluation of Any Segmentation in Medical Images](https://mlsanity.com/paper/2409.14874/) (2024)
- [The Missing Piece: A Case for Pre-Training in 3D Medical Object Detection](https://mlsanity.com/paper/2509.15947/) (2025)
- [Rule-based outlier detection of AI-generated anatomy segmentations](https://mlsanity.com/paper/2406.14486/) (2024)

## Abstract (arXiv)

Purpose AI-based methods for anatomy segmentation can help automate characterization of large imaging datasets. The growing number of similar in functionality models raises the challenge of evaluating them on datasets that do not contain ground truth annotations. We introduce a practical framework to assist in this task. Approach We harmonize the segmentation results into a standard, interoperable representation, which enables consistent, terminology-based labeling of the structures. We extend 3D Slicer to streamline loading and comparison of these harmonized segmentations, and demonstrate how standard representation simplifies review of the results using interactive summary plots and browser-based visualization using OHIF Viewer. To demonstrate the utility of the approach we apply it to evaluating segmentation of 31 anatomical structures (lungs, vertebrae, ribs, and heart) by six open-source models - TotalSegmentator 1.5 and 2.6, Auto3DSeg, MOOSE, MultiTalent, and CADS - for a sample of Computed Tomography (CT) scans from the publicly available National Lung Screening Trial (NLST) dataset. Results We demonstrate the utility of the framework in enabling automating loading, structure-wise inspection and comparison across models. Preliminary results ascertain practical utility of the approach in allowing quick detection and review of problematic results. The comparison shows excellent agreement segmenting some (e.g., lung) but not all structures (e.g., some models produce invalid vertebrae or rib segmentations). Conclusions The resources developed are linked from https://imagingdatacommons.github.io/segmentation-comparison/ including segmentation harmonization scripts, summary plots, and visualization tools. This work assists in model evaluation in absence of ground truth, ultimately enabling informed model selection.