# Deep learning-based transformation of the H&E stain into special stains

arXiv: https://arxiv.org/abs/2008.08871 · HTML: https://mlsanity.com/paper/2008.08871/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2020.

## Built on by (most cited first, 30 of 55)

- [FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings](https://mlsanity.com/paper/2210.04620/) (2022)
- [Deep Learning-enabled Virtual Histological Staining of Biological Samples](https://mlsanity.com/paper/2211.06822/) (2022)
- [Domain Generalization in Computational Pathology: Survey and Guidelines](https://mlsanity.com/paper/2310.19656/) (2023)
- [Multistain Pretraining for Slide Representation Learning in Pathology](https://mlsanity.com/paper/2408.02859/) (2024)
- [Label-free virtual HER2 immunohistochemical staining of breast tissue using deep learning](https://mlsanity.com/paper/2112.05240/) (2021)
- [Deep Learning for Computational Cytology: A Survey](https://mlsanity.com/paper/2202.05126/) (2022)
- [Self-supervised learning of hologram reconstruction using physics consistency](https://mlsanity.com/paper/2209.08288/) (2022)
- [Virtual stain transfer in histology via cascaded deep neural networks](https://mlsanity.com/paper/2207.06578/) (2022)
- [Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions](https://mlsanity.com/paper/2304.06662/) (2023)
- [The ACROBAT 2022 Challenge: Automatic Registration Of Breast Cancer Tissue](https://mlsanity.com/paper/2305.18033/) (2023)
- [Virtual histological staining of unlabeled autopsy tissue](https://mlsanity.com/paper/2308.00920/) (2023)
- [Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancement](https://mlsanity.com/paper/2411.09894/) (2024)
- [ACROBAT -- a multi-stain breast cancer histological whole-slide-image data set from routine diagnostics for computational pathology](https://mlsanity.com/paper/2211.13621/) (2022)
- [Demonstration of an Adversarial Attack Against a Multimodal Vision Language Model for Pathology Imaging](https://mlsanity.com/paper/2401.02565/) (2024)
- [Virtual birefringence imaging and histological staining of amyloid deposits in label-free tissue using autofluorescence microscopy and deep learning](https://mlsanity.com/paper/2403.09100/) (2024)
- [Pixel super-resolved virtual staining of label-free tissue using diffusion models](https://mlsanity.com/paper/2410.20073/) (2024)
- [Improving Unsupervised Stain-To-Stain Translation using Self-Supervision and Meta-Learning](https://mlsanity.com/paper/2112.08837/) (2021)
- [Few-shot Transfer Learning for Holographic Image Reconstruction using a Recurrent Neural Network](https://mlsanity.com/paper/2201.11333/) (2022)
- [Digital staining in optical microscopy using deep learning -- a review](https://mlsanity.com/paper/2303.08140/) (2023)
- [A comparative evaluation of image-to-image translation methods for stain transfer in histopathology](https://mlsanity.com/paper/2303.17009/) (2023)
- [Analysis and Validation of Image Search Engines in Histopathology](https://mlsanity.com/paper/2401.03271/) (2024)
- [A robust and scalable framework for hallucination detection in virtual tissue staining and digital pathology](https://mlsanity.com/paper/2404.18458/) (2024)
- [Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization](https://mlsanity.com/paper/2407.04245/) (2024)
- [Generating Seamless Virtual Immunohistochemical Whole Slide Images with Content and Color Consistency](https://mlsanity.com/paper/2410.01072/) (2024)
- [Unpaired Multi-Domain Histopathology Virtual Staining using Dual Path Prompted Inversion](https://mlsanity.com/paper/2412.11106/) (2024)
- [Ultra-high-resolution unpaired stain transformation via Kernelized Instance Normalization](https://mlsanity.com/paper/2208.10730/) (2022)
- [A Multi-Stage Auto-Context Deep Learning Framework for Tissue and Nuclei Segmentation and Classification in H&E-Stained Histological Images of Advanced Melanoma](https://mlsanity.com/paper/2503.23958/) (2025)
- [AV-GAN: Attention-Based Varifocal Generative Adversarial Network for Uneven Medical Image Translation](https://mlsanity.com/paper/2404.10714/) (2024)
- [SHAP-CAT: A interpretable multi-modal framework enhancing WSI classification via virtual staining and shapley-value-based multimodal fusion](https://mlsanity.com/paper/2410.01408/) (2024)
- [USIGAN: Unbalanced Self-Information Feature Transport for Weakly Paired Image IHC Virtual Staining](https://mlsanity.com/paper/2507.05843/) (2025)

## Abstract (arXiv)

Pathology is practiced by visual inspection of histochemically stained slides. Most commonly, the hematoxylin and eosin (H&E) stain is used in the diagnostic workflow and it is the gold standard for cancer diagnosis. However, in many cases, especially for non-neoplastic diseases, additional "special stains" are used to provide different levels of contrast and color to tissue components and allow pathologists to get a clearer diagnostic picture. In this study, we demonstrate the utility of supervised learning-based computational stain transformation from H&E to different special stains (Masson's Trichrome, periodic acid-Schiff and Jones silver stain) using tissue sections from kidney needle core biopsies. Based on evaluation by three renal pathologists, followed by adjudication by a fourth renal pathologist, we show that the generation of virtual special stains from existing H&E images improves the diagnosis in several non-neoplastic kidney diseases sampled from 58 unique subjects. A second study performed by three pathologists found that the quality of the special stains generated by the stain transformation network was statistically equivalent to those generated through standard histochemical staining. As the transformation of H&E images into special stains can be achieved within 1 min or less per patient core specimen slide, this stain-to-stain transformation framework can improve the quality of the preliminary diagnosis when additional special stains are needed, along with significant savings in time and cost, reducing the burden on healthcare system and patients.