# Controllable Diffusion-Based Lesion Inpainting for Scalable Histopathology Data Augmentation

arXiv: https://arxiv.org/abs/2601.08127 · HTML: https://mlsanity.com/paper/2601.08127/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 13 Jan 2026.

## Builds on (most cited first, 17 of 17)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation](https://mlsanity.com/paper/2312.05239/) (2023)
- [CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models](https://mlsanity.com/paper/2407.15886/) (2024)
- [Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Study](https://mlsanity.com/paper/2311.00265/) (2023)
- [Domain Generalization for Medical Image Analysis: A Review](https://mlsanity.com/paper/2310.08598/) (2023)
- [Learn like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification](https://mlsanity.com/paper/2009.13698/) (2020)
- [Image Data Augmentation Approaches: A Comprehensive Survey and Future directions](https://mlsanity.com/paper/2301.02830/) (2023)
- [Generative Adversarial Networks in Digital Pathology: A Survey on Trends and Future Potential](https://mlsanity.com/paper/2004.14936/) (2020)
- [Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks](https://mlsanity.com/paper/2401.07709/) (2024)
- [Generative AI Enables Medical Image Segmentation in Ultra Low-Data Regimes](https://mlsanity.com/paper/2408.17421/) (2024)
- [KPIs 2024 Challenge: Advancing Glomerular Segmentation from Patch- to Slide-Level](https://mlsanity.com/paper/2502.07288/) (2025)

## Abstract (arXiv)

Expert-annotated training data remains the critical bottleneck for AI in histopathology, particularly for rare pathologies where even dozens of cases may be unavailable. While data augmentation offers a solution, existing methods fail to generate sufficiently realistic lesion morphologies that preserve tissue-specific architectures. Here we present PathoGen, a diffusion-based generative model enabling controllable, high-fidelity lesion inpainting into benign histopathology images. We validate PathoGen across four datasets representing kidney, skin, breast, and prostate pathology. Quantitative assessment confirms PathoGen outperforms state-of-the-art baselines in image fidelity and distributional similarity. Evaluation by six expert pathologists revealed that synthetic images by PathoGen were only marginally distinguished from real tissue image slightly above chance (57.75% accuracy), demonstrating strong perceptual realism of PathoGen-generated lesions. PathoGen achieved the highest win rate (35.4%) when pathologists ranked generation quality against all baselines. Crucially, augmenting training sets with PathoGen-synthesized lesions improves segmentation Dice scores by up to 0.18 compared to traditional augmentations, with maximum benefit in data-scarce regimes. By simultaneously generating realistic morphology and pixel-level annotations, PathoGen effectively addresses both data scarcity and annotation cost, two critical bottlenecks in computational pathology development.