# 3D-UCaps: 3D Capsules Unet for Volumetric Image Segmentation

arXiv: https://arxiv.org/abs/2203.08965 · HTML: https://mlsanity.com/paper/2203.08965/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 15 of 15)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation](https://mlsanity.com/paper/1611.09326/) (2016)
- [CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation](https://mlsanity.com/paper/1806.04051/) (2018)
- [Capsules for Object Segmentation](https://mlsanity.com/paper/1804.04241/) (2018)
- [Improving the Robustness of Capsule Networks to Image Affine Transformations](https://mlsanity.com/paper/1911.07968/) (2019)
- [Capsules for Biomedical Image Segmentation](https://mlsanity.com/paper/2004.04736/) (2020)
- [Capsule Network is Not More Robust than Convolutional Network](https://mlsanity.com/paper/2103.15459/) (2021)
- [Offset Curves Loss for Imbalanced Problem in Medical Segmentation](https://mlsanity.com/paper/2012.02463/) (2020)
- [A Multi-task Contextual Atrous Residual Network for Brain Tumor Detection & Segmentation](https://mlsanity.com/paper/2012.02073/) (2020)
- [Deep Recurrent Level Set for Segmenting Brain Tumors](https://mlsanity.com/paper/1810.04752/) (2018)

## Built on by (most cited first, 7 of 7)

- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [MEGANet: Multi-Scale Edge-Guided Attention Network for Weak Boundary Polyp Segmentation](https://mlsanity.com/paper/2309.03329/) (2023)
- [EmbryosFormer: Deformable Transformer and Collaborative Encoding-Decoding for Embryos Stage Development Classification](https://mlsanity.com/paper/2210.04615/) (2022)
- [SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction](https://mlsanity.com/paper/2309.03335/) (2023)
- [I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnoses](https://mlsanity.com/paper/2309.13550/) (2023)
- [Conditional diffusion model with spatial attention and latent embedding for medical image segmentation](https://mlsanity.com/paper/2502.06997/) (2025)
- [3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation](https://mlsanity.com/paper/2205.09299/) (2022)

## Abstract (arXiv)

Medical image segmentation has been so far achieving promising results with Convolutional Neural Networks (CNNs). However, it is arguable that in traditional CNNs, its pooling layer tends to discard important information such as positions. Moreover, CNNs are sensitive to rotation and affine transformation. Capsule network is a data-efficient network design proposed to overcome such limitations by replacing pooling layers with dynamic routing and convolutional strides, which aims to preserve the part-whole relationships. Capsule network has shown a great performance in image recognition and natural language processing, but applications for medical image segmentation, particularly volumetric image segmentation, has been limited. In this work, we propose 3D-UCaps, a 3D voxel-based Capsule network for medical volumetric image segmentation. We build the concept of capsules into a CNN by designing a network with two pathways: the first pathway is encoded by 3D Capsule blocks, whereas the second pathway is decoded by 3D CNNs blocks. 3D-UCaps, therefore inherits the merits from both Capsule network to preserve the spatial relationship and CNNs to learn visual representation. We conducted experiments on various datasets to demonstrate the robustness of 3D-UCaps including iSeg-2017, LUNA16, Hippocampus, and Cardiac, where our method outperforms previous Capsule networks and 3D-Unets.