# The Fully Convolutional Transformer for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2206.00566 · HTML: https://mlsanity.com/paper/2206.00566/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 1 Jun 2022.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.03024/) (2021)
- [DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation](https://mlsanity.com/paper/2106.06716/) (2021)
- [Mixed Transformer U-Net For Medical Image Segmentation](https://mlsanity.com/paper/2111.04734/) (2021)
- [LeViT-UNet: Make Faster Encoders with Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2107.08623/) (2021)
- [U-Net Transformer: Self and Cross Attention for Medical Image Segmentation](https://mlsanity.com/paper/2103.06104/) (2021)
- [TransClaw U-Net: Claw U-Net with Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2107.05188/) (2021)
- [Boundary-aware Transformers for Skin Lesion Segmentation](https://mlsanity.com/paper/2110.03864/) (2021)
- [D-Former: A U-shaped Dilated Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2201.00462/) (2022)
- [FocusNet: An attention-based Fully Convolutional Network for Medical Image Segmentation](https://mlsanity.com/paper/1902.03091/) (2019)
- [FatNet: A Feature-attentive Network for 3D Point Cloud Processing](https://mlsanity.com/paper/2104.03427/) (2021)
- [CpT: Convolutional Point Transformer for 3D Point Cloud Processing](https://mlsanity.com/paper/2111.10866/) (2021)
- [SAWNet: A Spatially Aware Deep Neural Network for 3D Point Cloud Processing](https://mlsanity.com/paper/1905.07650/) (2019)
- [Automatic Myocardial Infarction Evaluation from Delayed-Enhancement Cardiac MRI using Deep Convolutional Networks](https://mlsanity.com/paper/2010.16198/) (2020)
- [Penalizing small errors using an Adaptive Logarithmic Loss](https://mlsanity.com/paper/1910.09717/) (2019)

## Built on by (most cited first, 30 of 32)

- [TP-DRSeg: Improving Diabetic Retinopathy Lesion Segmentation with Explicit Text-Prompts Assisted SAM](https://mlsanity.com/paper/2406.15764/) (2024)
- [Optimizing Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2210.08066/) (2022)
- [Attention Mechanisms in Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.17937/) (2023)
- [Vision Transformers in Medical Imaging: A Review](https://mlsanity.com/paper/2211.10043/) (2022)
- [Dilated-UNet: A Fast and Accurate Medical Image Segmentation Approach using a Dilated Transformer and U-Net Architecture](https://mlsanity.com/paper/2304.11450/) (2023)
- [One registration is worth two segmentations](https://mlsanity.com/paper/2405.10879/) (2024)
- [GenSelfDiff-HIS: Generative Self-Supervision Using Diffusion for Histopathological Image Segmentation](https://mlsanity.com/paper/2309.01487/) (2023)
- [Deep models for stroke segmentation: do complex architectures always perform better?](https://mlsanity.com/paper/2403.17177/) (2024)
- [SAMReg: SAM-enabled Image Registration with ROI-based Correspondence](https://mlsanity.com/paper/2410.14083/) (2024)
- [One-shot and Partially-Supervised Cell Image Segmentation Using Small Visual Prompt](https://mlsanity.com/paper/2304.07991/) (2023)
- [GLFNET: Global-Local (frequency) Filter Networks for efficient medical image segmentation](https://mlsanity.com/paper/2403.00396/) (2024)
- [GLIMS: Attention-Guided Lightweight Multi-Scale Hybrid Network for Volumetric Semantic Segmentation](https://mlsanity.com/paper/2404.17854/) (2024)
- [Manifold-Aware Local Feature Modeling for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2410.10287/) (2024)
- [Dual Interaction Network with Cross-Image Attention for Medical Image Segmentation](https://mlsanity.com/paper/2509.05953/) (2025)
- [Using Reed-Muller Codes for Classification with Rejection and Recovery](https://mlsanity.com/paper/2309.06359/) (2023)
- [TransONet: Automatic Segmentation of Vasculature in Computed Tomographic Angiograms Using Deep Learning](https://mlsanity.com/paper/2311.10328/) (2023)
- [Automated Measurement of Vascular Calcification in Femoral Endarterectomy Patients Using Deep Learning](https://mlsanity.com/paper/2311.16001/) (2023)
- [Attention-Mamba: A Mamba-Enhanced Multi-Scale Parallel Inference Network for Medical Image Segmentation](https://mlsanity.com/paper/2402.02286/) (2024)
- [Uncertainty-driven and Adversarial Calibration Learning for Epicardial Adipose Tissue Segmentation](https://mlsanity.com/paper/2402.14349/) (2024)
- [CFPFormer: Feature-pyramid like Transformer Decoder for Segmentation and Detection](https://mlsanity.com/paper/2404.15451/) (2024)
- [IGAF: Incremental Guided Attention Fusion for Depth Super-Resolution](https://mlsanity.com/paper/2501.01723/) (2025)
- [Uncertainty-Based Ensemble Learning in CMR Semantic Segmentation](https://mlsanity.com/paper/2502.09269/) (2025)
- [Co-Evidential Fusion with Information Volume for Medical Image Segmentation](https://mlsanity.com/paper/2506.02492/) (2025)
- [InceptionMamba: Efficient Multi-Stage Feature Enhancement with Selective State Space Model for Microscopic Medical Image Segmentation](https://mlsanity.com/paper/2506.12208/) (2025)
- [Deeply Dual Supervised learning for melanoma recognition](https://mlsanity.com/paper/2508.01994/) (2025)
- [Edge Detection for Organ Boundaries via Top Down Refinement and SubPixel Upsampling](https://mlsanity.com/paper/2508.06805/) (2025)
- [DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentation](https://mlsanity.com/paper/2508.06816/) (2025)
- [VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagation](https://mlsanity.com/paper/2508.06819/) (2025)
- [CLFSeg: A Fuzzy-Logic based Solution for Boundary Clarity and Uncertainty Reduction in Medical Image Segmentation](https://mlsanity.com/paper/2510.24202/) (2025)
- [SPG-CDENet: Spatial Prior-Guided Cross Dual Encoder Network for Multi-Organ Segmentation](https://mlsanity.com/paper/2510.26390/) (2025)

## Abstract (arXiv)

We propose a novel transformer model, capable of segmenting medical images of varying modalities. Challenges posed by the fine grained nature of medical image analysis mean that the adaptation of the transformer for their analysis is still at nascent stages. The overwhelming success of the UNet lay in its ability to appreciate the fine-grained nature of the segmentation task, an ability which existing transformer based models do not currently posses. To address this shortcoming, we propose The Fully Convolutional Transformer (FCT), which builds on the proven ability of Convolutional Neural Networks to learn effective image representations, and combines them with the ability of Transformers to effectively capture long-term dependencies in its inputs. The FCT is the first fully convolutional Transformer model in medical imaging literature. It processes its input in two stages, where first, it learns to extract long range semantic dependencies from the input image, and then learns to capture hierarchical global attributes from the features. FCT is compact, accurate and robust. Our results show that it outperforms all existing transformer architectures by large margins across multiple medical image segmentation datasets of varying data modalities without the need for any pre-training. FCT outperforms its immediate competitor on the ACDC dataset by 1.3%, on the Synapse dataset by 4.4%, on the Spleen dataset by 1.2% and on ISIC 2017 dataset by 1.1% on the dice metric, with up to five times fewer parameters. Our code, environments and models will be available via GitHub.