# Hierarchical 3D fully convolutional networks for multi-organ segmentation

arXiv: https://arxiv.org/abs/1704.06382 · HTML: https://mlsanity.com/paper/1704.06382/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2017.

## Built on by (most cited first, 20 of 20)

- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation](https://mlsanity.com/paper/2212.04497/) (2022)
- [SegFormer3D: an Efficient Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2404.10156/) (2024)
- [Deep Learning Based 3D Segmentation: A Survey](https://mlsanity.com/paper/2103.05423/) (2021)
- [Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks](https://mlsanity.com/paper/2410.02331/) (2024)
- [Medical Image Segmentation with 3D Convolutional Neural Networks: A Survey](https://mlsanity.com/paper/2108.08467/) (2021)
- [Tackling the Class Imbalance Problem of Deep Learning Based Head and Neck Organ Segmentation](https://mlsanity.com/paper/2201.01636/) (2022)
- [Towards more precise automatic analysis: a comprehensive survey of deep learning-based multi-organ segmentation](https://mlsanity.com/paper/2303.00232/) (2023)
- [HALOS: Hallucination-free Organ Segmentation after Organ Resection Surgery](https://mlsanity.com/paper/2303.07717/) (2023)
- [AgileFormer: Spatially Agile Transformer UNet for Medical Image Segmentation](https://mlsanity.com/paper/2404.00122/) (2024)
- [Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation](https://mlsanity.com/paper/2302.03296/) (2023)
- [Multi-organ Segmentation Network with Adversarial Performance Validator](https://mlsanity.com/paper/2204.07850/) (2022)
- [TSUBF-Net: Trans-Spatial UNet-like Network with Bi-direction Fusion for Segmentation of Adenoid Hypertrophy in CT](https://mlsanity.com/paper/2412.00787/) (2024)
- [Three-dimensional visualization of X-ray micro-CT with large-scale datasets: Efficiency and accuracy for real-time interaction](https://mlsanity.com/paper/2601.15098/) (2026)
- [Prior Guided Deep Difference Meta-Learner for Fast Adaptation to Stylized Segmentation](https://mlsanity.com/paper/2211.10588/) (2022)
- [Detection and Segmentation of Pancreas using Morphological Snakes and Deep Convolutional Neural Networks](https://mlsanity.com/paper/2302.06356/) (2023)
- [Learnable Weight Initialization for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2306.09320/) (2023)
- [SAS: Segment Anything Small for Ultrasound -- A Non-Generative Data Augmentation Technique for Robust Deep Learning in Ultrasound Imaging](https://mlsanity.com/paper/2503.05916/) (2025)
- [SPG-CDENet: Spatial Prior-Guided Cross Dual Encoder Network for Multi-Organ Segmentation](https://mlsanity.com/paper/2510.26390/) (2025)
- [Multi-planar 2D-U-Net Segmentation of 3D-CT Abdominal Organs augmented by Spatial Occurrence Maps](https://mlsanity.com/paper/2606.07717/) (2026)

## Abstract (arXiv)

Recent advances in 3D fully convolutional networks (FCN) have made it feasible to produce dense voxel-wise predictions of full volumetric images. In this work, we show that a multi-class 3D FCN trained on manually labeled CT scans of seven abdominal structures (artery, vein, liver, spleen, stomach, gallbladder, and pancreas) can achieve competitive segmentation results, while avoiding the need for handcrafting features or training organ-specific models. To this end, we propose a two-stage, coarse-to-fine approach that trains an FCN model to roughly delineate the organs of interest in the first stage (seeing ~40% of the voxels within a simple, automatically generated binary mask of the patient's body). We then use these predictions of the first-stage FCN to define a candidate region that will be used to train a second FCN. This step reduces the number of voxels the FCN has to classify to ~10% while maintaining a recall high of >99%. This second-stage FCN can now focus on more detailed segmentation of the organs. We respectively utilize training and validation sets consisting of 281 and 50 clinical CT images. Our hierarchical approach provides an improved Dice score of 7.5 percentage points per organ on average in our validation set. We furthermore test our models on a completely unseen data collection acquired at a different hospital that includes 150 CT scans with three anatomical labels (liver, spleen, and pancreas). In such challenging organs as the pancreas, our hierarchical approach improves the mean Dice score from 68.5 to 82.2%, achieving the highest reported average score on this dataset.