# Diverse Image Generation with Diffusion Models and Cross Class Label Learning for Polyp Classification

arXiv: https://arxiv.org/abs/2502.05444 · HTML: https://mlsanity.com/paper/2502.05444/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 8 Feb 2025.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Ambiguous Medical Image Segmentation using Diffusion Models](https://mlsanity.com/paper/2304.04745/) (2023)
- [ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models](https://mlsanity.com/paper/2309.01111/) (2023)
- [GastroVision: A Multi-class Endoscopy Image Dataset for Computer Aided Gastrointestinal Disease Detection](https://mlsanity.com/paper/2307.08140/) (2023)
- [Mask-conditioned latent diffusion for generating gastrointestinal polyp images](https://mlsanity.com/paper/2304.05233/) (2023)
- [Medical image reconstruction with image-adaptive priors learned by use of generative adversarial networks](https://mlsanity.com/paper/2001.10830/) (2020)
- [How Can We Make GAN Perform Better in Single Medical Image Super-Resolution? A Lesion Focused Multi-Scale Approach](https://mlsanity.com/paper/1901.03419/) (2019)
- [GAN Inversion for Data Augmentation to Improve Colonoscopy Lesion Classification](https://mlsanity.com/paper/2205.02840/) (2022)
- [PolypConnect: Image inpainting for generating realistic gastrointestinal tract images with polyps](https://mlsanity.com/paper/2205.15413/) (2022)
- [Can Adversarial Networks Make Uninformative Colonoscopy Video Frames Clinically Informative?](https://mlsanity.com/paper/2304.02152/) (2023)

## Built on by (most cited first, 3 of 3)

- [C3VDv2 -- Colonoscopy 3D video dataset with enhanced realism](https://mlsanity.com/paper/2506.24074/) (2025)
- [Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2505.05573/) (2025)
- [ColoDiff: Integrating Dynamic Consistency With Content Awareness for Colonoscopy Video Generation](https://mlsanity.com/paper/2602.23203/) (2026)

## Abstract (arXiv)

Pathologic diagnosis is a critical phase in deciding the optimal treatment procedure for dealing with colorectal cancer (CRC). Colonic polyps, precursors to CRC, can pathologically be classified into two major types: adenomatous and hyperplastic. For precise classification and early diagnosis of such polyps, the medical procedure of colonoscopy has been widely adopted paired with various imaging techniques, including narrow band imaging and white light imaging. However, the existing classification techniques mainly rely on a single imaging modality and show limited performance due to data scarcity. Recently, generative artificial intelligence has been gaining prominence in overcoming such issues. Additionally, various generation-controlling mechanisms using text prompts and images have been introduced to obtain visually appealing and desired outcomes. However, such mechanisms require class labels to make the model respond efficiently to the provided control input. In the colonoscopy domain, such controlling mechanisms are rarely explored; specifically, the text prompt is a completely uninvestigated area. Moreover, the unavailability of expensive class-wise labels for diverse sets of images limits such explorations. Therefore, we develop a novel model, PathoPolyp-Diff, that generates text-controlled synthetic images with diverse characteristics in terms of pathology, imaging modalities, and quality. We introduce cross-class label learning to make the model learn features from other classes, reducing the burdensome task of data annotation. The experimental results report an improvement of up to 7.91% in balanced accuracy using a publicly available dataset. Moreover, cross-class label learning achieves a statistically significant improvement of up to 18.33% in balanced accuracy during video-level analysis. The code is available at https://github.com/Vanshali/PathoPolyp-Diff.