# FCN-Transformer Feature Fusion for Polyp Segmentation

arXiv: https://arxiv.org/abs/2208.08352 · HTML: https://mlsanity.com/paper/2208.08352/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 17 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Road Extraction by Deep Residual U-Net](https://mlsanity.com/paper/1711.10684/) (2017)
- [Highway Networks](https://mlsanity.com/paper/1505.00387/) (2015)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [ResUNet++: An Advanced Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1911.07067/) (2019)
- [DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation](https://mlsanity.com/paper/2006.04868/) (2020)
- [HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS](https://mlsanity.com/paper/2101.07172/) (2021)
- [UACANet: Uncertainty Augmented Context Attention for Polyp Segmentation](https://mlsanity.com/paper/2107.02368/) (2021)
- [Stepwise Feature Fusion: Local Guides Global](https://mlsanity.com/paper/2203.03635/) (2022)
- [MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation](https://mlsanity.com/paper/2105.07451/) (2021)
- [DDANet: Dual Decoder Attention Network for Automatic Polyp Segmentation](https://mlsanity.com/paper/2012.15245/) (2020)

## Built on by (most cited first, 30 of 37)

- [Using DUCK-Net for Polyp Image Segmentation](https://mlsanity.com/paper/2311.02239/) (2023)
- [SimCol3D -- 3D Reconstruction during Colonoscopy Challenge](https://mlsanity.com/paper/2307.11261/) (2023)
- [A Survey on Deep Learning for Polyp Segmentation: Techniques, Challenges and Future Trends](https://mlsanity.com/paper/2311.18373/) (2023)
- [Contextrast: Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2404.10633/) (2024)
- [Attention Mechanisms in Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.17937/) (2023)
- [RWKV-UNet: Improving UNet with Long-Range Cooperation for Effective Medical Image Segmentation](https://mlsanity.com/paper/2501.08458/) (2025)
- [Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registration](https://mlsanity.com/paper/2509.06467/) (2025)
- [FCB-SwinV2 Transformer for Polyp Segmentation](https://mlsanity.com/paper/2302.01027/) (2023)
- [Colorectal Polyp Segmentation in the Deep Learning Era: A Comprehensive Survey](https://mlsanity.com/paper/2401.11734/) (2024)
- [Part-aware Prompted Segment Anything Model for Adaptive Segmentation](https://mlsanity.com/paper/2403.05433/) (2024)
- [MNet-SAt: A Multiscale Network with Spatial-enhanced Attention for Segmentation of Polyps in Colonoscopy](https://mlsanity.com/paper/2412.19464/) (2024)
- [D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse](https://mlsanity.com/paper/2506.14107/) (2025)
- [Multi-scale Information Sharing and Selection Network with Boundary Attention for Polyp Segmentation](https://mlsanity.com/paper/2405.11151/) (2024)
- [LAPFormer: A Light and Accurate Polyp Segmentation Transformer](https://mlsanity.com/paper/2210.04393/) (2022)
- [PAMI: partition input and aggregate outputs for model interpretation](https://mlsanity.com/paper/2302.03318/) (2023)
- [A Comparison of Deep Learning Architectures for Spacecraft Anomaly Detection](https://mlsanity.com/paper/2403.12864/) (2024)
- [A Point-Neighborhood Learning Framework for Nasal Endoscope Image Segmentation](https://mlsanity.com/paper/2405.20044/) (2024)
- [S2S2: Semantic Stacking for Robust Semantic Segmentation in Medical Imaging](https://mlsanity.com/paper/2412.13156/) (2024)
- [WoundAmbit: Bridging State-of-the-Art Semantic Segmentation and Real-World Wound Care](https://mlsanity.com/paper/2504.06185/) (2025)
- [Interpretable Small Training Set Image Segmentation Network Originated from Multi-Grid Variational Model](https://mlsanity.com/paper/2306.14097/) (2023)
- [CMFDNet: Cross-Mamba and Feature Discovery Network for Polyp Segmentation](https://mlsanity.com/paper/2508.17729/) (2025)
- [DSNet: a simple yet efficient network with dual-stream attention for lesion segmentation](https://mlsanity.com/paper/2211.16950/) (2022)
- [Prototype Learning for Out-of-Distribution Polyp Segmentation](https://mlsanity.com/paper/2308.03709/) (2023)
- [Benchmarking Feature Extractors for Reinforcement Learning-Based Semiconductor Defect Localization](https://mlsanity.com/paper/2311.11145/) (2023)
- [Polyp Segmentation Generalisability of Pretrained Backbones](https://mlsanity.com/paper/2405.15524/) (2024)
- [H-FCBFormer Hierarchical Fully Convolutional Branch Transformer for Occlusal Contact Segmentation with Articulating Paper](https://mlsanity.com/paper/2407.07604/) (2024)
- [Ensemble architecture in polyp segmentation](https://mlsanity.com/paper/2408.07262/) (2024)
- [Multi-scale and Multi-path Cascaded Convolutional Network for Semantic Segmentation of Colorectal Polyps](https://mlsanity.com/paper/2412.02443/) (2024)
- [MedSegNet10: A Publicly Accessible Network Repository for Split Federated Medical Image Segmentation](https://mlsanity.com/paper/2503.20830/) (2025)
- [Semi-Supervised Semantic Segmentation via Derivative Label Propagation](https://mlsanity.com/paper/2508.02254/) (2025)

## Abstract (arXiv)

Colonoscopy is widely recognised as the gold standard procedure for the early detection of colorectal cancer (CRC). Segmentation is valuable for two significant clinical applications, namely lesion detection and classification, providing means to improve accuracy and robustness. The manual segmentation of polyps in colonoscopy images is time-consuming. As a result, the use of deep learning (DL) for automation of polyp segmentation has become important. However, DL-based solutions can be vulnerable to overfitting and the resulting inability to generalise to images captured by different colonoscopes. Recent transformer-based architectures for semantic segmentation both achieve higher performance and generalise better than alternatives, however typically predict a segmentation map of frach4×fracw4 spatial dimensions for a h× w input image. To this end, we propose a new architecture for full-size segmentation which leverages the strengths of a transformer in extracting the most important features for segmentation in a primary branch, while compensating for its limitations in full-size prediction with a secondary fully convolutional branch. The resulting features from both branches are then fused for final prediction of a h× w segmentation map. We demonstrate our method's state-of-the-art performance with respect to the mDice, mIoU, mPrecision, and mRecall metrics, on both the Kvasir-SEG and CVC-ClinicDB dataset benchmarks. Additionally, we train the model on each of these datasets and evaluate on the other to demonstrate its superior generalisation performance.