# Polyp-E: Benchmarking the Robustness of Deep Segmentation Models via Polyp Editing

arXiv: https://arxiv.org/abs/2410.16732 · HTML: https://mlsanity.com/paper/2410.16732/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Builds on (most cited first, 30 of 30)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images](https://mlsanity.com/paper/1612.00799/) (2016)
- [Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers](https://mlsanity.com/paper/2108.06932/) (2021)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [On the Robustness of Semantic Segmentation Models to Adversarial Attacks](https://mlsanity.com/paper/1711.09856/) (2017)
- [MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data](https://mlsanity.com/paper/2002.03366/) (2020)
- [Reverse Attention for Salient Object Detection](https://mlsanity.com/paper/1807.09940/) (2018)
- [Shallow Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2108.00882/) (2021)
- [Transformer for Graphs: An Overview from Architecture Perspective](https://mlsanity.com/paper/2202.08455/) (2022)
- [Video Polyp Segmentation: A Deep Learning Perspective](https://mlsanity.com/paper/2203.14291/) (2022)
- [Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train](https://mlsanity.com/paper/2306.16741/) (2023)
- [Polyp-SAM: Transfer SAM for Polyp Segmentation](https://mlsanity.com/paper/2305.00293/) (2023)
- [UACANet: Uncertainty Augmented Context Attention for Polyp Segmentation](https://mlsanity.com/paper/2107.02368/) (2021)
- [ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models](https://mlsanity.com/paper/2309.01111/) (2023)
- [ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing](https://mlsanity.com/paper/2303.17096/) (2023)
- [Assessing generalisability of deep learning-based polyp detection and segmentation methods through a computer vision challenge](https://mlsanity.com/paper/2202.12031/) (2022)
- [M2SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation](https://mlsanity.com/paper/2303.10894/) (2023)
- [ROOD-MRI: Benchmarking the robustness of deep learning segmentation models to out-of-distribution and corrupted data in MRI](https://mlsanity.com/paper/2203.06060/) (2022)
- [Robustness Stress Testing in Medical Image Classification](https://mlsanity.com/paper/2308.06889/) (2023)
- [PP-SAM: Perturbed Prompts for Robust Adaptation of Segment Anything Model for Polyp Segmentation](https://mlsanity.com/paper/2405.16740/) (2024)

## Abstract (arXiv)

Automatic polyp segmentation is helpful to assist clinical diagnosis and treatment. In daily clinical practice, clinicians exhibit robustness in identifying polyps with both location and size variations. It is uncertain if deep segmentation models can achieve comparable robustness in automated colonoscopic analysis. To benchmark the model robustness, we focus on evaluating the robustness of segmentation models on the polyps with various attributes (e.g. location and size) and healthy samples. Based on the Latent Diffusion Model, we perform attribute editing on real polyps and build a new dataset named Polyp-E. Our synthetic dataset boasts exceptional realism, to the extent that clinical experts find it challenging to discern them from real data. We evaluate several existing polyp segmentation models on the proposed benchmark. The results reveal most of the models are highly sensitive to attribute variations. As a novel data augmentation technique, the proposed editing pipeline can improve both in-distribution and out-of-distribution generalization ability. The code and datasets will be released.