# Modality specific U-Net variants for biomedical image segmentation: A survey

arXiv: https://arxiv.org/abs/2107.04537 · HTML: https://mlsanity.com/paper/2107.04537/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 97 of its 233 references that are in the map.
- First posted on arXiv 9 Jul 2021.

## Builds on (most cited first, 30 of 97)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)

## Built on by (most cited first, 20 of 20)

- [Learning Deep Intensity Field for Extremely Sparse-View CBCT Reconstruction](https://mlsanity.com/paper/2303.06681/) (2023)
- [A Survey on Training Challenges in Generative Adversarial Networks for Biomedical Image Analysis](https://mlsanity.com/paper/2201.07646/) (2022)
- [BT-Unet: A self-supervised learning framework for biomedical image segmentation using Barlow Twins with U-Net models](https://mlsanity.com/paper/2112.03916/) (2021)
- [MAG-Net: Multi-task attention guided network for brain tumor segmentation and classification](https://mlsanity.com/paper/2107.12321/) (2021)
- [Impact of Attention on Adversarial Robustness of Image Classification Models](https://mlsanity.com/paper/2109.00936/) (2021)
- [Deep Learning Architecture Based Approach For 2D-Simulation of Microwave Plasma Interaction](https://mlsanity.com/paper/2206.01263/) (2022)
- [Hybrid Skip: A Biologically Inspired Skip Connection for the UNet Architecture](https://mlsanity.com/paper/2207.04721/) (2022)
- [Segmentation of Prostate Tumour Volumes from PET Images is a Different Ball Game](https://mlsanity.com/paper/2407.10537/) (2024)
- [Biomedical Image Segmentation: A Systematic Literature Review of Deep Learning Based Object Detection Methods](https://mlsanity.com/paper/2408.03393/) (2024)
- [Temporal Saliency Detection Towards Explainable Transformer-based Timeseries Forecasting](https://mlsanity.com/paper/2212.07771/) (2022)
- [Impact of the composition of feature extraction and class sampling in medicare fraud detection](https://mlsanity.com/paper/2206.01413/) (2022)
- [IODeep: an IOD for the introduction of deep learning in the DICOM standard](https://mlsanity.com/paper/2311.16163/) (2023)
- [Aortic root landmark localization with optimal transport loss for heatmap regression](https://mlsanity.com/paper/2407.04921/) (2024)
- [Polyp segmentation in colonoscopy images using DeepLabV3++](https://mlsanity.com/paper/2407.19327/) (2024)
- [SYNAPSE-Net: A Unified Framework with Lesion-Aware Hierarchical Gating for Robust Segmentation of Heterogeneous Brain Lesions](https://mlsanity.com/paper/2510.26961/) (2025)
- [MedSAM-based lung masking for multi-label chest X-ray classification](https://mlsanity.com/paper/2512.23089/) (2025)
- [An Intuitionistic Fuzzy Logic Driven UNet architecture: Application to Brain Image segmentation](https://mlsanity.com/paper/2602.04227/) (2026)
- [A Novel Framework using Intuitionistic Fuzzy Logic with U-Net and U-Net++ Architecture: A case Study of MRI Bain Image Segmentation](https://mlsanity.com/paper/2603.18042/) (2026)
- [An Uncertainty-Aware Loss Function Incorporating Fuzzy Logic: Application to MRI Brain Image Segmentation](https://mlsanity.com/paper/2604.16490/) (2026)
- [Compute-Optimal Network Design for Echocardiography Myocardial Segmentation and Perfusion Quantification using Neural Scaling Laws](https://mlsanity.com/paper/2606.06725/) (2026)

## Abstract (arXiv)

With the advent of advancements in deep learning approaches, such as deep convolution neural network, residual neural network, adversarial network; U-Net architectures are most widely utilized in biomedical image segmentation to address the automation in identification and detection of the target regions or sub-regions. In recent studies, U-Net based approaches have illustrated state-of-the-art performance in different applications for the development of computer-aided diagnosis systems for early diagnosis and treatment of diseases such as brain tumor, lung cancer, alzheimer, breast cancer, etc., using various modalities. This article contributes in presenting the success of these approaches by describing the U-Net framework, followed by the comprehensive analysis of the U-Net variants by performing 1) inter-modality, and 2) intra-modality categorization to establish better insights into the associated challenges and solutions. Besides, this article also highlights the contribution of U-Net based frameworks in the ongoing pandemic, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) also known as COVID-19. Finally, the strengths and similarities of these U-Net variants are analysed along with the challenges involved in biomedical image segmentation to uncover promising future research directions in this area.