# GAN Inversion for Data Augmentation to Improve Colonoscopy Lesion Classification

arXiv: https://arxiv.org/abs/2205.02840 · HTML: https://mlsanity.com/paper/2205.02840/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 4 May 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Embedding Propagation: Smoother Manifold for Few-Shot Classification](https://mlsanity.com/paper/2003.04151/) (2020)
- [Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training](https://mlsanity.com/paper/1711.06606/) (2017)
- [DU-GAN: Generative Adversarial Networks with Dual-Domain U-Net Based Discriminators for Low-Dose CT Denoising](https://mlsanity.com/paper/2108.10772/) (2021)
- [Learning Signal-Agnostic Manifolds of Neural Fields](https://mlsanity.com/paper/2111.06387/) (2021)
- [Colonoscopy Polyp Detection and Classification: Dataset Creation and Comparative Evaluations](https://mlsanity.com/paper/2104.10824/) (2021)
- [Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation](https://mlsanity.com/paper/2003.12473/) (2020)
- [Automatic Colon Polyp Detection using Region based Deep CNN and Post Learning Approaches](https://mlsanity.com/paper/1906.11463/) (2019)
- [Improving colonoscopy lesion classification using semi-supervised deep learning](https://mlsanity.com/paper/2009.03162/) (2020)
- [Improving Endoscopic Decision Support Systems by Translating Between Imaging Modalities](https://mlsanity.com/paper/2004.12604/) (2020)

## Built on by (most cited first, 7 of 7)

- [Intra-Source Style Augmentation for Improved Domain Generalization](https://mlsanity.com/paper/2210.10175/) (2022)
- [C3VDv2 -- Colonoscopy 3D video dataset with enhanced realism](https://mlsanity.com/paper/2506.24074/) (2025)
- [Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization](https://mlsanity.com/paper/2307.00648/) (2023)
- [Diverse Image Generation with Diffusion Models and Cross Class Label Learning for Polyp Classification](https://mlsanity.com/paper/2502.05444/) (2025)
- [MeVGAN: GAN-based Plugin Model for Video Generation with Applications in Colonoscopy](https://mlsanity.com/paper/2311.03884/) (2023)
- [Prediction of Rectal Cancer Regrowth from Longitudinal Endoscopy](https://mlsanity.com/paper/2605.12855/) (2026)
- [C3VD-DEFCOL: A Deformable Colonoscopy Dataset with Time-Resolved 3D Ground Truth and Realistic Appearance](https://mlsanity.com/paper/2606.07891/) (2026)

## Abstract (arXiv)

A major challenge in applying deep learning to medical imaging is the paucity of annotated data. This study demonstrates that synthetic colonoscopy images generated by Generative Adversarial Network (GAN) inversion can be used as training data to improve the lesion classification performance of deep learning models. This approach inverts pairs of images with the same label to a semantically rich & disentangled latent space and manipulates latent representations to produce new synthetic images with the same label. We perform image modality translation (style transfer) between white light and narrowband imaging (NBI). We also generate realistic-looking synthetic lesion images by interpolating between original training images to increase the variety of lesion shapes in the training dataset. We show that these approaches outperform comparative colonoscopy data augmentation techniques without the need to re-train multiple generative models. This approach also leverages information from datasets that may not have been designed for the specific colonoscopy downstream task. E.g. using a bowel prep grading dataset for a polyp classification task. Our experiments show this approach can perform multiple colonoscopy data augmentations, which improve the downstream polyp classification performance over baseline and comparison methods by up to 6%.