# Capsules for Biomedical Image Segmentation

arXiv: https://arxiv.org/abs/2004.04736 · HTML: https://mlsanity.com/paper/2004.04736/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2020.

## Built on by (most cited first, 18 of 18)

- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [Salient Object Detection via Integrity Learning](https://mlsanity.com/paper/2101.07663/) (2021)
- [Beyond Self-Attention: Deformable Large Kernel Attention for Medical Image Segmentation](https://mlsanity.com/paper/2309.00121/) (2023)
- [High-Definition Map Generation Technologies For Autonomous Driving](https://mlsanity.com/paper/2206.05400/) (2022)
- [Clustering Propagation for Universal Medical Image Segmentation](https://mlsanity.com/paper/2403.16646/) (2024)
- [SS-3DCapsNet: Self-supervised 3D Capsule Networks for Medical Segmentation on Less Labeled Data](https://mlsanity.com/paper/2201.05905/) (2022)
- [Learning with Capsules: A Survey](https://mlsanity.com/paper/2206.02664/) (2022)
- [3D-UCaps: 3D Capsules Unet for Volumetric Image Segmentation](https://mlsanity.com/paper/2203.08965/) (2022)
- [Routing with Self-Attention for Multimodal Capsule Networks](https://mlsanity.com/paper/2112.00775/) (2021)
- [3DConvCaps: 3DUnet with Convolutional Capsule Encoder for Medical Image Segmentation](https://mlsanity.com/paper/2205.09299/) (2022)
- [Multi-Contrast MRI Segmentation Trained on Synthetic Images](https://mlsanity.com/paper/2207.02469/) (2022)
- [The Past, Current, and Future of Neonatal Intensive Care Units with Artificial Intelligence](https://mlsanity.com/paper/2302.00225/) (2023)
- [Retinal Vessel Segmentation with Deep Graph and Capsule Reasoning](https://mlsanity.com/paper/2409.11508/) (2024)
- [Selecting the Best Optimizers for Deep Learning based Medical Image Segmentation](https://mlsanity.com/paper/2302.02289/) (2023)
- [CapsNet for Medical Image Segmentation](https://mlsanity.com/paper/2203.08948/) (2022)
- [Wound Segmentation with Dynamic Illumination Correction and Dual-view Semantic Fusion](https://mlsanity.com/paper/2207.05388/) (2022)
- [P-NOC: adversarial training of CAM generating networks for robust weakly supervised semantic segmentation priors](https://mlsanity.com/paper/2305.12522/) (2023)
- [3D-EffiViTCaps: 3D Efficient Vision Transformer with Capsule for Medical Image Segmentation](https://mlsanity.com/paper/2403.16350/) (2024)

## Abstract (arXiv)

Our work expands the use of capsule networks to the task of object segmentation for the first time in the literature. This is made possible via the introduction of locally-constrained routing and transformation matrix sharing, which reduces the parameter/memory burden and allows for the segmentation of objects at large resolutions. To compensate for the loss of global information in constraining the routing, we propose the concept of "deconvolutional" capsules to create a deep encoder-decoder style network, called SegCaps. We extend the masked reconstruction regularization to the task of segmentation and perform thorough ablation experiments on each component of our method. The proposed convolutional-deconvolutional capsule network, SegCaps, shows state-of-the-art results while using a fraction of the parameters of popular segmentation networks. To validate our proposed method, we perform experiments segmenting pathological lungs from clinical and pre-clinical thoracic computed tomography (CT) scans and segmenting muscle and adipose (fat) tissue from magnetic resonance imaging (MRI) scans of human subjects' thighs. Notably, our experiments in lung segmentation represent the largest-scale study in pathological lung segmentation in the literature, where we conduct experiments across five extremely challenging datasets, containing both clinical and pre-clinical subjects, and nearly 2000 computed-tomography scans. Our newly developed segmentation platform outperforms other methods across all datasets while utilizing less than 5% of the parameters in the popular U-Net for biomedical image segmentation. Further, we demonstrate capsules' ability to generalize to unseen rotations/reflections on natural images.