# ColonFormer: An Efficient Transformer based Method for Colon Polyp Segmentation

arXiv: https://arxiv.org/abs/2205.08473 · HTML: https://mlsanity.com/paper/2205.08473/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 60 references that are in the map.
- First posted on arXiv 17 May 2022.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016)
- [ResUNet++: An Advanced Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1911.07067/) (2019)
- [MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation](https://mlsanity.com/paper/1902.04049/) (2019)
- [F3Net: Fusion, Feedback and Focus for Salient Object Detection](https://mlsanity.com/paper/1911.11445/) (2019)

## Built on by (most cited first, 28 of 28)

- [Using DUCK-Net for Polyp Image Segmentation](https://mlsanity.com/paper/2311.02239/) (2023)
- [TransNetR: Transformer-based Residual Network for Polyp Segmentation with Multi-Center Out-of-Distribution Testing](https://mlsanity.com/paper/2303.07428/) (2023)
- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [A Survey on Deep Learning for Polyp Segmentation: Techniques, Challenges and Future Trends](https://mlsanity.com/paper/2311.18373/) (2023)
- [Contextrast: Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2404.10633/) (2024)
- [Attention Mechanisms in Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.17937/) (2023)
- [Polyper: Boundary Sensitive Polyp Segmentation](https://mlsanity.com/paper/2312.08735/) (2023)
- [EffiSegNet: Gastrointestinal Polyp Segmentation through a Pre-Trained EfficientNet-based Network with a Simplified Decoder](https://mlsanity.com/paper/2407.16298/) (2024)
- [Polyp-DAM: Polyp segmentation via depth anything model](https://mlsanity.com/paper/2402.02298/) (2024)
- [LAPFormer: A Light and Accurate Polyp Segmentation Transformer](https://mlsanity.com/paper/2210.04393/) (2022)
- [UGCANet: A Unified Global Context-Aware Transformer-based Network with Feature Alignment for Endoscopic Image Analysis](https://mlsanity.com/paper/2307.06260/) (2023)
- [PolypFlow: Reinforcing Polyp Segmentation with Flow-Driven Dynamics](https://mlsanity.com/paper/2502.19037/) (2025)
- [Online pseudo labeling for polyp segmentation with momentum networks](https://mlsanity.com/paper/2209.14599/) (2022)
- [DCELANM-Net:Medical Image Segmentation based on Dual Channel Efficient Layer Aggregation Network with Learner](https://mlsanity.com/paper/2304.09620/) (2023)
- [TransRUPNet for Improved Polyp Segmentation](https://mlsanity.com/paper/2306.02176/) (2023)
- [HiFiSeg: High-Frequency Information Enhanced Polyp Segmentation with Global-Local Vision Transformer](https://mlsanity.com/paper/2410.02528/) (2024)
- [Automated GI tract segmentation using deep learning](https://mlsanity.com/paper/2206.11048/) (2022)
- [Rethinking the transfer learning for FCN based polyp segmentation in colonoscopy](https://mlsanity.com/paper/2211.02416/) (2022)
- [Polyp segmentation in colonoscopy images using DeepLabV3++](https://mlsanity.com/paper/2407.19327/) (2024)
- [Multi-scale and Multi-path Cascaded Convolutional Network for Semantic Segmentation of Colorectal Polyps](https://mlsanity.com/paper/2412.02443/) (2024)
- [MedSegNet10: A Publicly Accessible Network Repository for Split Federated Medical Image Segmentation](https://mlsanity.com/paper/2503.20830/) (2025)
- [FocusNet: Transformer-enhanced Polyp Segmentation with Local and Pooling Attention](https://mlsanity.com/paper/2504.13597/) (2025)
- [Hybrid(Transformer+CNN)-based Polyp Segmentation](https://mlsanity.com/paper/2508.09189/) (2025)
- [PVTAdpNet: Polyp Segmentation using Pyramid vision transformer with a novel Adapter block](https://mlsanity.com/paper/2509.23751/) (2025)
- [CLFSeg: A Fuzzy-Logic based Solution for Boundary Clarity and Uncertainty Reduction in Medical Image Segmentation](https://mlsanity.com/paper/2510.24202/) (2025)
- [A multi-center analysis of deep learning methods for video polyp detection and segmentation](https://mlsanity.com/paper/2603.04288/) (2026)
- [Metrics or Mirage? An Audit of Evaluation Inconsistencies in Colonoscopy Polyp Segmentation Benchmarks](https://mlsanity.com/paper/2607.08203/) (2026)

## Abstract (arXiv)

Identifying polyps is challenging for automatic analysis of endoscopic images in computer-aided clinical support systems. Models based on convolutional networks (CNN), transformers, and their combinations have been proposed to segment polyps with promising results. However, those approaches have limitations either in modeling the local appearance of the polyps only or lack of multi-level features for spatial dependency in the decoding process. This paper proposes a novel network, namely ColonFormer, to address these limitations. ColonFormer is an encoder-decoder architecture capable of modeling long-range semantic information at both encoder and decoder branches. The encoder is a lightweight architecture based on transformers for modeling global semantic relations at multi scales. The decoder is a hierarchical network structure designed for learning multi-level features to enrich feature representation. Besides, a refinement module is added with a new skip connection technique to refine the boundary of polyp objects in the global map for accurate segmentation. Extensive experiments have been conducted on five popular benchmark datasets for polyp segmentation, including Kvasir, CVC-Clinic DB, CVC-ColonDB, CVC-T, and ETIS-Larib. Experimental results show that our ColonFormer outperforms other state-of-the-art methods on all benchmark datasets.