# PathoPainter: Augmenting Histopathology Segmentation via Tumor-aware Inpainting

arXiv: https://arxiv.org/abs/2503.04634 · HTML: https://mlsanity.com/paper/2503.04634/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 25 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Builds on (most cited first, 20 of 20)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [CAMEL: A Weakly Supervised Learning Framework for Histopathology Image Segmentation](https://mlsanity.com/paper/1908.10555/) (2019)
- [Towards Generalizable Tumor Synthesis](https://mlsanity.com/paper/2402.19470/) (2024)
- [FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models](https://mlsanity.com/paper/2310.15160/) (2023)
- [HookNet: multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images](https://mlsanity.com/paper/2006.12230/) (2020)
- [Multi-Layer Pseudo-Supervision for Histopathology Tissue Semantic Segmentation using Patch-level Classification Labels](https://mlsanity.com/paper/2110.08048/) (2021)
- [Learned representation-guided diffusion models for large-image generation](https://mlsanity.com/paper/2312.07330/) (2023)
- [DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology](https://mlsanity.com/paper/2306.13384/) (2023)
- [Fast and Accurate Tumor Segmentation of Histology Images using Persistent Homology and Deep Convolutional Features](https://mlsanity.com/paper/1805.03699/) (2018)
- [Pan-tumor CAnine cuTaneous Cancer Histology (CATCH) dataset](https://mlsanity.com/paper/2201.11446/) (2022)
- [WSI-SAM: Multi-resolution Segment Anything Model (SAM) for histopathology whole-slide images](https://mlsanity.com/paper/2403.09257/) (2024)
- [Deep Learning-Based Grading of Ductal Carcinoma In Situ in Breast Histopathology Images](https://mlsanity.com/paper/2010.03244/) (2020)
- [Style-Extracting Diffusion Models for Semi-Supervised Histopathology Segmentation](https://mlsanity.com/paper/2403.14429/) (2024)

## Built on by (most cited first, 3 of 3)

- [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025)
- [Video Generative Models as Geometry Learner](https://mlsanity.com/paper/2608.28549/) (2026)
- [Longitudinal Lesion Inpainting in Brain MRI via 3D Region Aware Diffusion](https://mlsanity.com/paper/2603.05693/) (2026)

## Abstract (arXiv)

Tumor segmentation plays a critical role in histopathology, but it requires costly, fine-grained image-mask pairs annotated by pathologists. Thus, synthesizing histopathology data to expand the dataset is highly desirable. Previous works suffer from inaccuracies and limited diversity in image-mask pairs, both of which affect training segmentation, particularly in small-scale datasets and the inherently complex nature of histopathology images. To address this challenge, we propose PathoPainter, which reformulates image-mask pair generation as a tumor inpainting task. Specifically, our approach preserves the background while inpainting the tumor region, ensuring precise alignment between the generated image and its corresponding mask. To enhance dataset diversity while maintaining biological plausibility, we incorporate a sampling mechanism that conditions tumor inpainting on regional embeddings from a different image. Additionally, we introduce a filtering strategy to exclude uncertain synthetic regions, further improving the quality of the generated data. Our comprehensive evaluation spans multiple datasets featuring diverse tumor types and various training data scales. As a result, segmentation improved significantly with our synthetic data, surpassing existing segmentation data synthesis approaches, e.g., 75.69% -> 77.69% on CAMELYON16. The code is available at https://github.com/HongLiuuuuu/PathoPainter.