# NeoUNet: Towards accurate colon polyp segmentation and neoplasm detection

arXiv: https://arxiv.org/abs/2107.05023 · HTML: https://mlsanity.com/paper/2107.05023/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 11 Jul 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation](https://mlsanity.com/paper/1912.05074/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Tversky loss function for image segmentation using 3D fully convolutional deep networks](https://mlsanity.com/paper/1706.05721/) (2017)
- [ResUNet++: An Advanced Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1911.07067/) (2019)
- [A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation](https://mlsanity.com/paper/1810.07842/) (2018)
- [DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation](https://mlsanity.com/paper/2006.04868/) (2020)
- [HarDNet: A Low Memory Traffic Network](https://mlsanity.com/paper/1909.00948/) (2019)
- [HarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS](https://mlsanity.com/paper/2101.07172/) (2021)
- [CU-Net: Coupled U-Nets](https://mlsanity.com/paper/1808.06521/) (2018)

## Built on by (most cited first, 30 of 51)

- [EMCAD: Efficient Multi-scale Convolutional Attention Decoding for Medical Image Segmentation](https://mlsanity.com/paper/2405.06880/) (2024)
- [TGANet: Text-guided attention for improved polyp segmentation](https://mlsanity.com/paper/2205.04280/) (2022)
- [Each Test Image Deserves A Specific Prompt: Continual Test-Time Adaptation for 2D Medical Image Segmentation](https://mlsanity.com/paper/2311.18363/) (2023)
- [ColonFormer: An Efficient Transformer based Method for Colon Polyp Segmentation](https://mlsanity.com/paper/2205.08473/) (2022)
- [Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models](https://mlsanity.com/paper/2308.07706/) (2023)
- [TransNetR: Transformer-based Residual Network for Polyp Segmentation with Multi-Center Out-of-Distribution Testing](https://mlsanity.com/paper/2303.07428/) (2023)
- [TransResU-Net: Transformer based ResU-Net for Real-Time Colonoscopy Polyp Segmentation](https://mlsanity.com/paper/2206.08985/) (2022)
- [Is SAM 2 Better than SAM in Medical Image Segmentation?](https://mlsanity.com/paper/2408.04212/) (2024)
- [Towards General Purpose Medical AI: Continual Learning Medical Foundation Model](https://mlsanity.com/paper/2303.06580/) (2023)
- [Frontiers in Intelligent Colonoscopy](https://mlsanity.com/paper/2410.17241/) (2024)
- [ProMamba: Prompt-Mamba for polyp segmentation](https://mlsanity.com/paper/2403.13660/) (2024)
- [VLSM-Adapter: Finetuning Vision-Language Segmentation Efficiently with Lightweight Blocks](https://mlsanity.com/paper/2405.06196/) (2024)
- [BlazeNeo: Blazing fast polyp segmentation and neoplasm detection](https://mlsanity.com/paper/2203.00129/) (2022)
- [UniBiomed: A Universal Foundation Model for Grounded Biomedical Image Interpretation](https://mlsanity.com/paper/2504.21336/) (2025)
- [Attention Mechanisms in Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.17937/) (2023)
- [MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning](https://mlsanity.com/paper/2602.03320/) (2026)
- [PolypDB: A Curated Multi-Center Dataset for Development of AI Algorithms in Colonoscopy](https://mlsanity.com/paper/2409.00045/) (2024)
- [SegCol Challenge: Semantic Segmentation for Tools and Fold Edges in Colonoscopy data](https://mlsanity.com/paper/2412.16078/) (2024)
- [DilatedSegNet: A Deep Dilated Segmentation Network for Polyp Segmentation](https://mlsanity.com/paper/2210.13595/) (2022)
- [Focal-UNet: UNet-like Focal Modulation for Medical Image Segmentation](https://mlsanity.com/paper/2212.09263/) (2022)
- [Language-guided Medical Image Segmentation with Target-informed Multi-level Contrastive Alignments](https://mlsanity.com/paper/2412.13533/) (2024)
- [Colorectal Polyp Segmentation in the Deep Learning Era: A Comprehensive Survey](https://mlsanity.com/paper/2401.11734/) (2024)
- [MedCLIPSeg: Probabilistic Vision-Language Adaptation for Data-Efficient and Generalizable Medical Image Segmentation](https://mlsanity.com/paper/2602.20423/) (2026)
- [M3FPolypSegNet: Segmentation Network with Multi-frequency Feature Fusion for Polyp Localization in Colonoscopy Images](https://mlsanity.com/paper/2310.05538/) (2023)
- [TuneVLSeg: Prompt Tuning Benchmark for Vision-Language Segmentation Models](https://mlsanity.com/paper/2410.05239/) (2024)
- [iDPA: Instance Decoupled Prompt Attention for Incremental Medical Object Detection](https://mlsanity.com/paper/2506.00406/) (2025)
- [EndoCogniAgent: Closed-Loop Agentic Reasoning with Self-Consistency Validation for Endoscopic Diagnosis](https://mlsanity.com/paper/2508.07292/) (2025)
- [K-Prism: A Knowledge-Guided and Prompt Integrated Universal Medical Image Segmentation Model](https://mlsanity.com/paper/2509.25594/) (2025)
- [LAPFormer: A Light and Accurate Polyp Segmentation Transformer](https://mlsanity.com/paper/2210.04393/) (2022)
- [RaBiT: An Efficient Transformer using Bidirectional Feature Pyramid Network with Reverse Attention for Colon Polyp Segmentation](https://mlsanity.com/paper/2307.06420/) (2023)

## Abstract (arXiv)

Automatic polyp segmentation has proven to be immensely helpful for endoscopy procedures, reducing the missing rate of adenoma detection for endoscopists while increasing efficiency. However, classifying a polyp as being neoplasm or not and segmenting it at the pixel level is still a challenging task for doctors to perform in a limited time. In this work, we propose a fine-grained formulation for the polyp segmentation problem. Our formulation aims to not only segment polyp regions, but also identify those at high risk of malignancy with high accuracy. In addition, we present a UNet-based neural network architecture called NeoUNet, along with a hybrid loss function to solve this problem. Experiments show highly competitive results for NeoUNet on our benchmark dataset compared to existing polyp segmentation models.