# Improving text-conditioned latent diffusion for cancer pathology

arXiv: https://arxiv.org/abs/2412.06487 · HTML: https://mlsanity.com/paper/2412.06487/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 6 of its 26 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 6 of 6)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later](https://mlsanity.com/paper/2303.02490/) (2023)
- [Unsupervised Histopathology Image Synthesis](https://mlsanity.com/paper/1712.05021/) (2017)
- [A Multi-attribute Controllable Generative Model for Histopathology Image Synthesis](https://mlsanity.com/paper/2111.06398/) (2021)
- [Augmenting Medical Imaging: A Comprehensive Catalogue of 65 Techniques for Enhanced Data Analysis](https://mlsanity.com/paper/2303.01178/) (2023)

## Built on by (most cited first, 2 of 2)

- [MSDM: Generating Task-Specific Pathology Images with a Multimodal Conditioned Diffusion Model for Cell and Nuclei Segmentation](https://mlsanity.com/paper/2510.09121/) (2025)
- [Central-to-Local Adaptive Generative Diffusion Framework for Improving Gene Expression Prediction in Data-Limited Spatial Transcriptomics](https://mlsanity.com/paper/2603.26827/) (2026)

## Abstract (arXiv)

The development of generative models in the past decade has allowed for hyperrealistic data synthesis. While potentially beneficial, this synthetic data generation process has been relatively underexplored in cancer histopathology. One algorithm for synthesising a realistic image is diffusion; it iteratively converts an image to noise and learns the recovery process from this noise [Wang and Vastola, 2023]. While effective, it is highly computationally expensive for high-resolution images, rendering it infeasible for histopathology. The development of Variational Autoencoders (VAEs) has allowed us to learn the representation of complex high-resolution images in a latent space. A vital by-product of this is the ability to compress high-resolution images to space and recover them lossless. The marriage of diffusion and VAEs allows us to carry out diffusion in the latent space of an autoencoder, enabling us to leverage the realistic generative capabilities of diffusion while maintaining reasonable computational requirements. Rombach et al. [2021b] and Yellapragada et al. [2023] build foundational models for this task, paving the way to generate realistic histopathology images. In this paper, we discuss the pitfalls of current methods, namely [Yellapragada et al., 2023] and resolve critical errors while proposing improvements along the way. Our methods achieve an FID score of 21.11, beating its SOTA counterparts in [Yellapragada et al., 2023] by 1.2 FID, while presenting a train-time GPU memory usage reduction of 7%.