# Mask-guided cross-image attention for zero-shot in-silico histopathologic image generation with a diffusion model arXiv: https://arxiv.org/abs/2407.11664 ยท HTML: https://mlsanity.com/paper/2407.11664/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. - Builds on 23 of its 25 references that are in the map. - First posted on arXiv 16 Jul 2024. ## Placed in - Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/) ## Builds on (most cited first, 23 of 23) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021) - [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022) - [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022) - [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021) - [Cross-Image Attention for Zero-Shot Appearance Transfer](https://mlsanity.com/paper/2311.03335/) (2023) - [A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images](https://mlsanity.com/paper/2209.13167/) (2022) - [Learning Vision from Models Rivals Learning Vision from Data](https://mlsanity.com/paper/2312.17742/) (2023) - [StainGAN: Stain Style Transfer for Digital Histological Images](https://mlsanity.com/paper/1804.01601/) (2018) - [PathLDM: Text conditioned Latent Diffusion Model for Histopathology](https://mlsanity.com/paper/2309.00748/) (2023) - [Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations](https://mlsanity.com/paper/2107.12357/) (2021) - [DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology](https://mlsanity.com/paper/2306.13384/) (2023) - [DiffMix: Diffusion Model-based Data Synthesis for Nuclei Segmentation and Classification in Imbalanced Pathology Image Datasets](https://mlsanity.com/paper/2306.14132/) (2023) - [A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images](https://mlsanity.com/paper/2208.14125/) (2022) - [NASDM: Nuclei-Aware Semantic Histopathology Image Generation Using Diffusion Models](https://mlsanity.com/paper/2303.11477/) (2023) - [Synthetic Augmentation with Large-scale Unconditional Pre-training](https://mlsanity.com/paper/2308.04020/) (2023) - [ViT-DAE: Transformer-driven Diffusion Autoencoder for Histopathology Image Analysis](https://mlsanity.com/paper/2304.01053/) (2023) - [StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining](https://mlsanity.com/paper/2403.11340/) (2024) - [Diffusion-based generation of Histopathological Whole Slide Images at a Gigapixel scale](https://mlsanity.com/paper/2311.08199/) (2023) - [Domain Adaptation-based Augmentation for Weakly Supervised Nuclei Detection](https://mlsanity.com/paper/1907.04681/) (2019) - [Recent Advances in Generative AI for Healthcare Applications](https://mlsanity.com/paper/2310.00795/) (2023) - [ReStainGAN: Leveraging IHC to IF Stain Domain Translation for in-silico Data Generation](https://mlsanity.com/paper/2403.06545/) (2024) - [Auxiliary CycleGAN-guidance for Task-Aware Domain Translation from Duplex to Monoplex IHC Images](https://mlsanity.com/paper/2403.07389/) (2024) ## Built on by (most cited first, 2 of 2) - [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025) - [Virtual Staining of Label-Free Tissue in Imaging Mass Spectrometry](https://mlsanity.com/paper/2411.13120/) (2024) ## Abstract (arXiv) Creating in-silico data with generative AI promises a cost-effective alternative to staining, imaging, and annotating whole slide images in computational pathology. Diffusion models are the state-of-the-art solution for generating in-silico images, offering unparalleled fidelity and realism. Using appearance transfer diffusion models allows for zero-shot image generation, facilitating fast application and making model training unnecessary. However current appearance transfer diffusion models are designed for natural images, where the main task is to transfer the foreground object from an origin to a target domain, while the background is of insignificant importance. In computational pathology, specifically in oncology, it is however not straightforward to define which objects in an image should be classified as foreground and background, as all objects in an image may be of critical importance for the detailed understanding the tumor micro-environment. We contribute to the applicability of appearance transfer diffusion models to immunohistochemistry-stained images by modifying the appearance transfer guidance to alternate between class-specific AdaIN feature statistics matchings using existing segmentation masks. The performance of the proposed method is demonstrated on the downstream task of supervised epithelium segmentation, showing that the number of manual annotations required for model training can be reduced by 75%, outperforming the baseline approach. Additionally, we consulted with a certified pathologist to investigate future improvements. We anticipate this work to inspire the application of zero-shot diffusion models in computational pathology, providing an efficient method to generate in-silico images with unmatched fidelity and realism, which prove meaningful for downstream tasks, such as training existing deep learning models or finetuning foundation models.