# A Comprehensive Study on Colorectal Polyp Segmentation with ResUNet++, Conditional Random Field and Test-Time Augmentation

arXiv: https://arxiv.org/abs/2107.12435 · HTML: https://mlsanity.com/paper/2107.12435/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 63 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Road Extraction by Deep Residual U-Net](https://mlsanity.com/paper/1711.10684/) (2017)
- [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016)
- [ResUNet++: An Advanced Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1911.07067/) (2019)
- [MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation](https://mlsanity.com/paper/1902.04049/) (2019)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation](https://mlsanity.com/paper/2006.04868/) (2020)
- [Polyp Segmentation in Colonoscopy Images Using Fully Convolutional Network](https://mlsanity.com/paper/1802.00368/) (2018)
- [Robust Medical Instrument Segmentation Challenge 2019](https://mlsanity.com/paper/2003.10299/) (2020)
- [Colorectal Polyp Segmentation by U-Net with Dilation Convolution](https://mlsanity.com/paper/1912.11947/) (2019)
- [Towards a Computed-Aided Diagnosis System in Colonoscopy: Automatic Polyp Segmentation Using Convolution Neural Networks](https://mlsanity.com/paper/2101.06040/) (2021)

## Built on by (most cited first, 30 of 41)

- [Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers](https://mlsanity.com/paper/2108.06932/) (2021)
- [TGANet: Text-guided attention for improved polyp segmentation](https://mlsanity.com/paper/2205.04280/) (2022)
- [A Comprehensive Survey of Foundation Models in Medicine](https://mlsanity.com/paper/2406.10729/) (2024)
- [OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining](https://mlsanity.com/paper/2411.15421/) (2024)
- [I-MedSAM: Implicit Medical Image Segmentation with Segment Anything](https://mlsanity.com/paper/2311.17081/) (2023)
- [Tyche: Stochastic In-Context Learning for Medical Image Segmentation](https://mlsanity.com/paper/2401.13650/) (2024)
- [Towards Diverse Binary Segmentation via A Simple yet General Gated Network](https://mlsanity.com/paper/2303.10396/) (2023)
- [Towards General Purpose Medical AI: Continual Learning Medical Foundation Model](https://mlsanity.com/paper/2303.06580/) (2023)
- [SMAFormer: Synergistic Multi-Attention Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2409.00346/) (2024)
- [Frontiers in Intelligent Colonoscopy](https://mlsanity.com/paper/2410.17241/) (2024)
- [A Survey on Deep Learning for Polyp Segmentation: Techniques, Challenges and Future Trends](https://mlsanity.com/paper/2311.18373/) (2023)
- [Polyp-DDPM: Diffusion-Based Semantic Polyp Synthesis for Enhanced Segmentation](https://mlsanity.com/paper/2402.04031/) (2024)
- [PolypDB: A Curated Multi-Center Dataset for Development of AI Algorithms in Colonoscopy](https://mlsanity.com/paper/2409.00045/) (2024)
- [Colorectal Polyp Segmentation in the Deep Learning Era: A Comprehensive Survey](https://mlsanity.com/paper/2401.11734/) (2024)
- [SwIPE: Efficient and Robust Medical Image Segmentation with Implicit Patch Embeddings](https://mlsanity.com/paper/2307.12429/) (2023)
- [Self-Supervised U-Net for Segmenting Flat and Sessile Polyps](https://mlsanity.com/paper/2110.08776/) (2021)
- [Membership Inference Attacks via Adversarial Examples](https://mlsanity.com/paper/2207.13572/) (2022)
- [Multi Kernel Positional Embedding ConvNeXt for Polyp Segmentation](https://mlsanity.com/paper/2301.06673/) (2023)
- [Cooperation Learning Enhanced Colonic Polyp Segmentation Based on Transformer-CNN Fusion](https://mlsanity.com/paper/2301.06892/) (2023)
- [Lagrange Duality and Compound Multi-Attention Transformer for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2409.07793/) (2024)
- [RetSeg: Retention-based Colorectal Polyps Segmentation Network](https://mlsanity.com/paper/2310.05446/) (2023)
- [A Medical Data-Effective Learning Benchmark for Highly Efficient Pre-training of Foundation Models](https://mlsanity.com/paper/2401.17542/) (2024)
- [A Lightweight and Robust Framework for Real-Time Colorectal Polyp Detection Using LOF-Based Preprocessing and YOLO-v11n](https://mlsanity.com/paper/2507.10864/) (2025)
- [A Tumor Aware DenseNet Swin Hybrid Learning with Boosted and Hierarchical Feature Spaces for Large-Scale Brain MRI Classification](https://mlsanity.com/paper/2601.18330/) (2026)
- [PAANet: Progressive Alternating Attention for Automatic Medical Image Segmentation](https://mlsanity.com/paper/2111.10618/) (2021)
- [How Low Can We Go? Pixel Annotation for Semantic Segmentation](https://mlsanity.com/paper/2201.10448/) (2022)
- [Rethinking the transfer learning for FCN based polyp segmentation in colonoscopy](https://mlsanity.com/paper/2211.02416/) (2022)
- [RUPNet: Residual upsampling network for real-time polyp segmentation](https://mlsanity.com/paper/2301.02703/) (2023)
- [Does image resolution impact chest X-ray based fine-grained Tuberculosis-consistent lesion segmentation?](https://mlsanity.com/paper/2301.04032/) (2023)
- [Detection of Pavement Cracks by Deep Learning Models of Transformer and UNet](https://mlsanity.com/paper/2304.12596/) (2023)

## Abstract (arXiv)

Colonoscopy is considered the gold standard for detection of colorectal cancer and its precursors. Existing examination methods are, however, hampered by high overall miss-rate, and many abnormalities are left undetected. Computer-Aided Diagnosis systems based on advanced machine learning algorithms are touted as a game-changer that can identify regions in the colon overlooked by the physicians during endoscopic examinations, and help detect and characterize lesions. In previous work, we have proposed the ResUNet++ architecture and demonstrated that it produces more efficient results compared with its counterparts U-Net and ResUNet. In this paper, we demonstrate that further improvements to the overall prediction performance of the ResUNet++ architecture can be achieved by using conditional random field and test-time augmentation. We have performed extensive evaluations and validated the improvements using six publicly available datasets: Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, ETIS-Larib Polyp DB, ASU-Mayo Clinic Colonoscopy Video Database, and CVC-VideoClinicDB. Moreover, we compare our proposed architecture and resulting model with other State-of-the-art methods. To explore the generalization capability of ResUNet++ on different publicly available polyp datasets, so that it could be used in a real-world setting, we performed an extensive cross-dataset evaluation. The experimental results show that applying CRF and TTA improves the performance on various polyp segmentation datasets both on the same dataset and cross-dataset.