# 3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2205.09299 · HTML: https://mlsanity.com/paper/2205.09299/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 30 of 32)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [CE-Net: Context Encoder Network for 2D Medical Image Segmentation](https://mlsanity.com/paper/1903.02740/) (2019)
- [The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes](https://mlsanity.com/paper/1904.00445/) (2019)
- [HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentation](https://mlsanity.com/paper/1804.02967/) (2018)
- [3D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Training](https://mlsanity.com/paper/1811.12506/) (2018)
- [3D Anisotropic Hybrid Network: Transferring Convolutional Features from 2D Images to 3D Anisotropic Volumes](https://mlsanity.com/paper/1711.08580/) (2017)
- [Brain Tumor Segmentation using an Ensemble of 3D U-Nets and Overall Survival Prediction using Radiomic Features](https://mlsanity.com/paper/1812.01049/) (2018)
- [Capsule Routing via Variational Bayes](https://mlsanity.com/paper/1905.11455/) (2019)
- [Group Equivariant Capsule Networks](https://mlsanity.com/paper/1806.05086/) (2018)
- [Capsules for Object Segmentation](https://mlsanity.com/paper/1804.04241/) (2018)
- [Improving the Robustness of Capsule Networks to Image Affine Transformations](https://mlsanity.com/paper/1911.07968/) (2019)
- [Capsules for Biomedical Image Segmentation](https://mlsanity.com/paper/2004.04736/) (2020)
- [Hierarchical Self-Supervised Learning for Medical Image Segmentation Based on Multi-Domain Data Aggregation](https://mlsanity.com/paper/2107.04886/) (2021)
- [Deep CNN ensembles and suggestive annotations for infant brain MRI segmentation](https://mlsanity.com/paper/1712.05319/) (2017)
- [Offset Curves Loss for Imbalanced Problem in Medical Segmentation](https://mlsanity.com/paper/2012.02463/) (2020)
- [Point-Unet: A Context-aware Point-based Neural Network for Volumetric Segmentation](https://mlsanity.com/paper/2203.08964/) (2022)
- [SS-3DCapsNet: Self-supervised 3D Capsule Networks for Medical Segmentation on Less Labeled Data](https://mlsanity.com/paper/2201.05905/) (2022)
- [Capsule Networks against Medical Imaging Data Challenges](https://mlsanity.com/paper/1807.07559/) (2018)
- [A Multi-task Contextual Atrous Residual Network for Brain Tumor Detection & Segmentation](https://mlsanity.com/paper/2012.02073/) (2020)
- [3D-UCaps: 3D Capsules Unet for Volumetric Image Segmentation](https://mlsanity.com/paper/2203.08965/) (2022)
- [Dilated Convolutions in Neural Networks for Left Atrial Segmentation in 3D Gadolinium Enhanced-MRI](https://mlsanity.com/paper/1808.01673/) (2018)
- [Deep Recurrent Level Set for Segmenting Brain Tumors](https://mlsanity.com/paper/1810.04752/) (2018)
- [DAM-AL: Dilated Attention Mechanism with Attention Loss for 3D Infant Brain Image Segmentation](https://mlsanity.com/paper/2112.13559/) (2021)

## Built on by (most cited first, 3 of 3)

- [I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnoses](https://mlsanity.com/paper/2309.13550/) (2023)
- [CarcassFormer: An End-to-end Transformer-based Framework for Simultaneous Localization, Segmentation and Classification of Poultry Carcass Defect](https://mlsanity.com/paper/2404.11429/) (2024)
- [3D-EffiViTCaps: 3D Efficient Vision Transformer with Capsule for Medical Image Segmentation](https://mlsanity.com/paper/2403.16350/) (2024)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) have achieved promising results in medical image segmentation. However, CNNs require lots of training data and are incapable of handling pose and deformation of objects. Furthermore, their pooling layers tend to discard important information such as positions as well as CNNs are sensitive to rotation and affine transformation. Capsule network is a recent new architecture that has achieved better robustness in part-whole representation learning by replacing pooling layers with dynamic routing and convolutional strides, which has shown potential results on popular tasks such as digit classification and object segmentation. In this paper, we propose a 3D encoder-decoder network with Convolutional Capsule Encoder (called 3DConvCaps) to learn lower-level features (short-range attention) with convolutional layers while modeling the higher-level features (long-range dependence) with capsule layers. Our experiments on multiple datasets including iSeg-2017, Hippocampus, and Cardiac demonstrate that our 3D 3DConvCaps network considerably outperforms previous capsule networks and 3D-UNets. We further conduct ablation studies of network efficiency and segmentation performance under various configurations of convolution layers and capsule layers at both contracting and expanding paths.