# VIMs: Virtual Immunohistochemistry Multiplex staining via Text-to-Stain Diffusion Trained on Uniplex Stains arXiv: https://arxiv.org/abs/2407.19113 ยท HTML: https://mlsanity.com/paper/2407.19113/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. - Builds on 19 of its 32 references that are in the map. - First posted on arXiv 26 Jul 2024. ## Placed in - Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/) ## Builds on (most cited first, 19 of 19) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016) - [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021) - [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023) - [One-Step Image Translation with Text-to-Image Models](https://mlsanity.com/paper/2403.12036/) (2024) - [Ensembling Off-the-shelf Models for GAN Training](https://mlsanity.com/paper/2112.09130/) (2021) - [Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue](https://mlsanity.com/paper/1803.11293/) (2018) - [StainGAN: Stain Style Transfer for Digital Histological Images](https://mlsanity.com/paper/1804.01601/) (2018) - [BCI: Breast Cancer Immunohistochemical Image Generation through Pyramid Pix2pix](https://mlsanity.com/paper/2204.11425/) (2022) - [PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning](https://mlsanity.com/paper/1807.07701/) (2018) - [StainNet: a fast and robust stain normalization network](https://mlsanity.com/paper/2012.12535/) (2020) - [GAN-based Virtual Re-Staining: A Promising Solution for Whole Slide Image Analysis](https://mlsanity.com/paper/1901.04059/) (2019) - [Structural Cycle GAN for Virtual Immunohistochemistry Staining of Gland Markers in the Colon](https://mlsanity.com/paper/2308.13182/) (2023) - [A comparative evaluation of image-to-image translation methods for stain transfer in histopathology](https://mlsanity.com/paper/2303.17009/) (2023) ## Built on by (most cited first, 10 of 10) - [Pixel super-resolved virtual staining of label-free tissue using diffusion models](https://mlsanity.com/paper/2410.20073/) (2024) - [HistDiST: Histopathological Diffusion-based Stain Transfer](https://mlsanity.com/paper/2505.06793/) (2025) - [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025) - [USIGAN: Unbalanced Self-Information Feature Transport for Weakly Paired Image IHC Virtual Staining](https://mlsanity.com/paper/2507.05843/) (2025) - [PGVMS: A Prompt-Guided Unified Framework for Virtual Multiplex IHC Staining with Pathological Semantic Learning](https://mlsanity.com/paper/2602.23292/) (2026) - [IMPLICITSTAINER: Resolution Agnostic Data-Efficient Virtual Staining Using Neural Implicit Functions](https://mlsanity.com/paper/2505.09831/) (2025) - [Virtual Multiplex Staining for Histological Images using a Marker-wise Conditioned Diffusion Model](https://mlsanity.com/paper/2508.14681/) (2025) - [Building Trust in Virtual Immunohistochemistry: Automated Assessment of Image Quality](https://mlsanity.com/paper/2511.04615/) (2025) - [Fast, Unsupervised Framework for Registration Quality Assessment of Multi-stain Histological Whole Slide Pairs](https://mlsanity.com/paper/2602.04046/) (2026) - [ContiStain: Cross-Domain Relation-Preserving Distillation for Continual Multi-Domain Virtual IHC Staining](https://mlsanity.com/paper/2607.03851/) (2026) ## Abstract (arXiv) This paper introduces a Virtual Immunohistochemistry Multiplex staining (VIMs) model designed to generate multiple immunohistochemistry (IHC) stains from a single hematoxylin and eosin (H&E) stained tissue section. IHC stains are crucial in pathology practice for resolving complex diagnostic questions and guiding patient treatment decisions. While commercial laboratories offer a wide array of up to 400 different antibody-based IHC stains, small biopsies often lack sufficient tissue for multiple stains while preserving material for subsequent molecular testing. This highlights the need for virtual IHC staining. Notably, VIMs is the first model to address this need, leveraging a large vision-language single-step diffusion model for virtual IHC multiplexing through text prompts for each IHC marker. VIMs is trained on uniplex paired H&E and IHC images, employing an adversarial training module. Testing of VIMs includes both paired and unpaired image sets. To enhance computational efficiency, VIMs utilizes a pre-trained large latent diffusion model fine-tuned with small, trainable weights through the Low-Rank Adapter (LoRA) approach. Experiments on nuclear and cytoplasmic IHC markers demonstrate that VIMs outperforms the base diffusion model and achieves performance comparable to Pix2Pix, a standard generative model for paired image translation. Multiple evaluation methods, including assessments by two pathologists, are used to determine the performance of VIMs. Additionally, experiments with different prompts highlight the impact of text conditioning. This paper represents the first attempt to accelerate histopathology research by demonstrating the generation of multiple IHC stains from a single H&E input using a single model trained solely on uniplex data.