# PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask Conditions

arXiv: https://arxiv.org/abs/2506.23440 · HTML: https://mlsanity.com/paper/2506.23440/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Quilt-1M: One Million Image-Text Pairs for Histopathology](https://mlsanity.com/paper/2306.11207/) (2023)
- [Deep neural network models for computational histopathology: A survey](https://mlsanity.com/paper/1912.12378/) (2019)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [CellViT: Vision Transformers for Precise Cell Segmentation and Classification](https://mlsanity.com/paper/2306.15350/) (2023)
- [Diffusion Probabilistic Models beat GANs on Medical Images](https://mlsanity.com/paper/2212.07501/) (2022)
- [A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images](https://mlsanity.com/paper/2209.13167/) (2022)
- [Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models](https://mlsanity.com/paper/2402.05210/) (2024)
- [PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathology](https://mlsanity.com/paper/2305.15072/) (2023)
- [Multiple Instance Learning for Digital Pathology: A Review on the State-of-the-Art, Limitations & Future Potential](https://mlsanity.com/paper/2206.04425/) (2022)
- [PathLDM: Text conditioned Latent Diffusion Model for Histopathology](https://mlsanity.com/paper/2309.00748/) (2023)
- [Learned representation-guided diffusion models for large-image generation](https://mlsanity.com/paper/2312.07330/) (2023)
- [Can segmentation models be trained with fully synthetically generated data?](https://mlsanity.com/paper/2209.08256/) (2022)
- [CoNIC Challenge: Pushing the Frontiers of Nuclear Detection, Segmentation, Classification and Counting](https://mlsanity.com/paper/2303.06274/) (2023)
- [PathologyGAN: Learning deep representations of cancer tissue](https://mlsanity.com/paper/1907.02644/) (2019)
- [Diffusion-based Data Augmentation for Nuclei Image Segmentation](https://mlsanity.com/paper/2310.14197/) (2023)
- [DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology](https://mlsanity.com/paper/2306.13384/) (2023)
- [DiffMix: Diffusion Model-based Data Synthesis for Nuclei Segmentation and Classification in Imbalanced Pathology Image Datasets](https://mlsanity.com/paper/2306.14132/) (2023)
- [Controllable and Efficient Multi-Class Pathology Nuclei Data Augmentation using Text-Conditioned Diffusion Models](https://mlsanity.com/paper/2407.14426/) (2024)
- [Co-synthesis of Histopathology Nuclei Image-Label Pairs using a Context-Conditioned Joint Diffusion Model](https://mlsanity.com/paper/2407.14434/) (2024)
- [Deep Learning Generates Synthetic Cancer Histology for Explainability and Education](https://mlsanity.com/paper/2211.06522/) (2022)
- [Histopathological Image Classification with Cell Morphology Aware Deep Neural Networks](https://mlsanity.com/paper/2407.08625/) (2024)

## Built on by (most cited first, 9 of 9)

- [Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype Control](https://mlsanity.com/paper/2512.21058/) (2025)
- [CytoDiff: AI-Driven Cytomorphology Image Synthesis for Medical Diagnostics](https://mlsanity.com/paper/2507.05063/) (2025)
- [Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers](https://mlsanity.com/paper/2603.13386/) (2026)
- [Score-Control for Hallucination Reduction in Diffusion Models](https://mlsanity.com/paper/2606.00377/) (2026)
- [PathAR: Structure-First Autoregressive Synthesis of Multimodal Pathology Images](https://mlsanity.com/paper/2606.01543/) (2026)
- [STREAM: Stochastic Riemannian Flow Matching with Anisotropic Decoder for Digital Histopathology Image Generation](https://mlsanity.com/paper/2606.07036/) (2026)
- [Controllable Histopathology Image Synthesis with Training-free Structural Initialization and Textural Modulation](https://mlsanity.com/paper/2606.27935/) (2026)
- [Destroy Me: Automatic Artifact Generation for Histopathology Images](https://mlsanity.com/paper/2608.27516/) (2026)
- [BigCell: Generating Gigapixel Whole-Slide Images](https://mlsanity.com/paper/2610.14261/) (2026)

## Abstract (arXiv)

Diffusion-based generative models have shown promise in synthesizing histopathology images to address data scarcity caused by privacy constraints. Diagnostic text reports provide high-level semantic descriptions, and masks offer fine-grained spatial structures essential for representing distinct morphological regions. However, public datasets lack paired text and mask data for the same histopathological images, limiting their joint use in image generation. This constraint restricts the ability to fully exploit the benefits of combining both modalities for enhanced control over semantics and spatial details. To overcome this, we propose PathDiff, a diffusion framework that effectively learns from unpaired mask-text data by integrating both modalities into a unified conditioning space. PathDiff allows precise control over structural and contextual features, generating high-quality, semantically accurate images. PathDiff also improves image fidelity, text-image alignment, and faithfulness, enhancing data augmentation for downstream tasks like nuclei segmentation and classification. Extensive experiments demonstrate its superiority over existing methods.