# Hierarchical 3D Feature Learning for Pancreas Segmentation

arXiv: https://arxiv.org/abs/2109.01667 · HTML: https://mlsanity.com/paper/2109.01667/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 3 Sep 2021.

## Builds on (most cited first, 12 of 12)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation](https://mlsanity.com/paper/1506.06448/) (2015)
- [Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation](https://mlsanity.com/paper/1702.00045/) (2017)
- [Deep Q Learning Driven CT Pancreas Segmentation with Geometry-Aware U-Net](https://mlsanity.com/paper/1904.09120/) (2019)
- [Spatial Aggregation of Holistically-Nested Networks for Automated Pancreas Segmentation](https://mlsanity.com/paper/1606.07830/) (2016)
- [INN: Inflated Neural Networks for IPMN Diagnosis](https://mlsanity.com/paper/1907.00437/) (2019)
- [Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function](https://mlsanity.com/paper/1707.04912/) (2017)
- [PAN: Projective Adversarial Network for Medical Image Segmentation](https://mlsanity.com/paper/1906.04378/) (2019)

## Built on by (most cited first, 6 of 6)

- [Dynamic Linear Transformer for 3D Biomedical Image Segmentation](https://mlsanity.com/paper/2206.00771/) (2022)
- [Radio astronomical images object detection and segmentation: A benchmark on deep learning methods](https://mlsanity.com/paper/2303.04506/) (2023)
- [Leveraging Anatomical Priors for Automated Pancreas Segmentation on Abdominal CT](https://mlsanity.com/paper/2504.06921/) (2025)
- [Cyst-X: A Multi-Center MRI Benchmark and Federated Learning Framework for Malignancy-Risk Stratification of Pancreatic Cystic Neoplasm](https://mlsanity.com/paper/2507.22017/) (2025)
- [Large-Scale Multi-Center CT and MRI Segmentation of Pancreas with Deep Learning](https://mlsanity.com/paper/2405.12367/) (2024)
- [Unified CT and MRI Pancreas Segmentation for Label-Efficient Cross-Modality Subregion Transfer](https://mlsanity.com/paper/2609.13043/) (2026)

## Abstract (arXiv)

We propose a novel 3D fully convolutional deep network for automated pancreas segmentation from both MRI and CT scans. More specifically, the proposed model consists of a 3D encoder that learns to extract volume features at different scales; features taken at different points of the encoder hierarchy are then sent to multiple 3D decoders that individually predict intermediate segmentation maps. Finally, all segmentation maps are combined to obtain a unique detailed segmentation mask. We test our model on both CT and MRI imaging data: the publicly available NIH Pancreas-CT dataset (consisting of 82 contrast-enhanced CTs) and a private MRI dataset (consisting of 40 MRI scans). Experimental results show that our model outperforms existing methods on CT pancreas segmentation, obtaining an average Dice score of about 88%, and yields promising segmentation performance on a very challenging MRI data set (average Dice score is about 77%). Additional control experiments demonstrate that the achieved performance is due to the combination of our 3D fully-convolutional deep network and the hierarchical representation decoding, thus substantiating our architectural design.