# Focal-UNet: UNet-like Focal Modulation for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2212.09263 · HTML: https://mlsanity.com/paper/2212.09263/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 19 Dec 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [Focal Self-attention for Local-Global Interactions in Vision Transformers](https://mlsanity.com/paper/2107.00641/) (2021)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [Focal Modulation Networks](https://mlsanity.com/paper/2203.11926/) (2022)
- [NeoUNet: Towards accurate colon polyp segmentation and neoplasm detection](https://mlsanity.com/paper/2107.05023/) (2021)
- [Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation](https://mlsanity.com/paper/2005.09120/) (2020)
- [U-SegNet: Fully Convolutional Neural Network based Automated Brain tissue segmentation Tool](https://mlsanity.com/paper/1806.04429/) (2018)
- [POCFormer: A Lightweight Transformer Architecture for Detection of COVID-19 Using Point of Care Ultrasound](https://mlsanity.com/paper/2105.09913/) (2021)

## Built on by (most cited first, 6 of 6)

- [BRAU-Net++: U-Shaped Hybrid CNN-Transformer Network for Medical Image Segmentation](https://mlsanity.com/paper/2401.00722/) (2024)
- [MCANet: Medical Image Segmentation with Multi-Scale Cross-Axis Attention](https://mlsanity.com/paper/2312.08866/) (2023)
- [CARE: A Large Scale CT Image Dataset and Clinical Applicable Benchmark Model for Rectal Cancer Segmentation](https://mlsanity.com/paper/2308.08283/) (2023)
- [MedFormer: Hierarchical Medical Vision Transformer with Content-Aware Dual Sparse Selection Attention](https://mlsanity.com/paper/2507.02488/) (2025)
- [Weakly supervised segmentation of intracranial aneurysms using a novel 3D focal modulation UNet](https://mlsanity.com/paper/2308.03001/) (2023)
- [RetSeg: Retention-based Colorectal Polyps Segmentation Network](https://mlsanity.com/paper/2310.05446/) (2023)

## Abstract (arXiv)

Recently, many attempts have been made to construct a transformer base U-shaped architecture, and new methods have been proposed that outperformed CNN-based rivals. However, serious problems such as blockiness and cropped edges in predicted masks remain because of transformers' patch partitioning operations. In this work, we propose a new U-shaped architecture for medical image segmentation with the help of the newly introduced focal modulation mechanism. The proposed architecture has asymmetric depths for the encoder and decoder. Due to the ability of the focal module to aggregate local and global features, our model could simultaneously benefit the wide receptive field of transformers and local viewing of CNNs. This helps the proposed method balance the local and global feature usage to outperform one of the most powerful transformer-based U-shaped models called Swin-UNet. We achieved a 1.68% higher DICE score and a 0.89 better HD metric on the Synapse dataset. Also, with extremely limited data, we had a 4.25% higher DICE score on the NeoPolyp dataset. Our implementations are available at: https://github.com/givkashi/Focal-UNet