# Towards Unifying Anatomy Segmentation: Automated Generation of a Full-body CT Dataset via Knowledge Aggregation and Anatomical Guidelines

arXiv: https://arxiv.org/abs/2307.13375 · HTML: https://mlsanity.com/paper/2307.13375/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Early-Learning Regularization Prevents Memorization of Noisy Labels](https://mlsanity.com/paper/2007.00151/) (2020)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?](https://mlsanity.com/paper/2010.14808/) (2020)
- [VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Images](https://mlsanity.com/paper/2001.09193/) (2020)
- [Label-Free Liver Tumor Segmentation](https://mlsanity.com/paper/2303.14869/) (2023)
- [Deep Learning to Segment Pelvic Bones: Large-scale CT Datasets and Baseline Models](https://mlsanity.com/paper/2012.08721/) (2020)
- [Omni-DETR: Omni-Supervised Object Detection with Transformers](https://mlsanity.com/paper/2203.16089/) (2022)
- [RibSeg Dataset and Strong Point Cloud Baselines for Rib Segmentation from CT Scans](https://mlsanity.com/paper/2109.09521/) (2021)
- [Deep Learning with Mixed Supervision for Brain Tumor Segmentation](https://mlsanity.com/paper/1812.04571/) (2018)
- [Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging](https://mlsanity.com/paper/2303.07126/) (2023)
- [Fully-automated Body Composition Analysis in Routine CT Imaging Using 3D Semantic Segmentation Convolutional Neural Networks](https://mlsanity.com/paper/2002.10776/) (2020)
- [Multi-site, Multi-domain Airway Tree Modeling (ATM'22): A Public Benchmark for Pulmonary Airway Segmentation](https://mlsanity.com/paper/2303.05745/) (2023)
- [One Weird Trick to Improve Your Semi-Weakly Supervised Semantic Segmentation Model](https://mlsanity.com/paper/2205.01233/) (2022)

## Built on by (most cited first, 17 of 17)

- [AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarking](https://mlsanity.com/paper/2407.16697/) (2024)
- [nnInteractive: Redefining 3D Promptable Segmentation](https://mlsanity.com/paper/2503.08373/) (2025)
- [MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision](https://mlsanity.com/paper/2308.16139/) (2023)
- [Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomography](https://mlsanity.com/paper/2405.18356/) (2024)
- [LesionLocator: Zero-Shot Universal Tumor Segmentation and Tracking in 3D Whole-Body Imaging](https://mlsanity.com/paper/2502.20985/) (2025)
- [Learning Segmentation from Radiology Reports](https://mlsanity.com/paper/2507.05582/) (2025)
- [Embracing Massive Medical Data](https://mlsanity.com/paper/2407.04687/) (2024)
- [Anatomy-guided Pathology Segmentation](https://mlsanity.com/paper/2407.05844/) (2024)
- [LOOC: Localizing Organs using Occupancy Networks and Body Surface Depth Images](https://mlsanity.com/paper/2406.12407/) (2024)
- [Anatomy Completor: A Multi-class Completion Framework for 3D Anatomy Reconstruction](https://mlsanity.com/paper/2309.04956/) (2023)
- [Quality Sentinel: Estimating Label Quality and Errors in Medical Segmentation Datasets](https://mlsanity.com/paper/2406.00327/) (2024)
- [Rethinking Whole-Body CT Image Interpretation: An Abnormality-Centric Approach](https://mlsanity.com/paper/2506.03238/) (2025)
- [Large-Scale Label Quality Assessment for Medical Segmentation via a Vision-Language Judge and Synthetic Data](https://mlsanity.com/paper/2601.14406/) (2026)
- [LIMIS: Towards Language-based Interactive Medical Image Segmentation](https://mlsanity.com/paper/2410.16939/) (2024)
- [State of Abdominal CT Datasets: A Critical Review of Bias, Clinical Relevance, and Real-world Applicability](https://mlsanity.com/paper/2508.13626/) (2025)
- [Modality-Projection Universal Model for Comprehensive Full-Body Medical Imaging Segmentation](https://mlsanity.com/paper/2412.19026/) (2024)
- [Large Scale Supervised Pretraining For Traumatic Brain Injury Segmentation](https://mlsanity.com/paper/2504.06741/) (2025)

## Abstract (arXiv)

In this study, we present a method for generating automated anatomy segmentation datasets using a sequential process that involves nnU-Net-based pseudo-labeling and anatomy-guided pseudo-label refinement. By combining various fragmented knowledge bases, we generate a dataset of whole-body CT scans with 142 voxel-level labels for 533 volumes providing comprehensive anatomical coverage which experts have approved. Our proposed procedure does not rely on manual annotation during the label aggregation stage. We examine its plausibility and usefulness using three complementary checks: Human expert evaluation which approved the dataset, a Deep Learning usefulness benchmark on the BTCV dataset in which we achieve 85% dice score without using its training dataset, and medical validity checks. This evaluation procedure combines scalable automated checks with labor-intensive high-quality expert checks. Besides the dataset, we release our trained unified anatomical segmentation model capable of predicting 142 anatomical structures on CT data.