# PAINT: Pathology-Aware Integrated Next-Scale Transformation for Virtual Immunohistochemistry

arXiv: https://arxiv.org/abs/2601.16024 · HTML: https://mlsanity.com/paper/2601.16024/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 22 Jan 2026.

## Builds on (most cited first, 28 of 28)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis](https://mlsanity.com/paper/2412.04431/) (2024)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [Deep neural network models for computational histopathology: A survey](https://mlsanity.com/paper/1912.12378/) (2019)
- [Distribution Matching Losses Can Hallucinate Features in Medical Image Translation](https://mlsanity.com/paper/1805.08841/) (2018)
- [On hallucinations in tomographic image reconstruction](https://mlsanity.com/paper/2012.00646/) (2020)
- [Adaptive Supervised PatchNCE Loss for Learning H&E-to-IHC Stain Translation with Inconsistent Groundtruth Image Pairs](https://mlsanity.com/paper/2303.06193/) (2023)
- [Hallucination Index: An Image Quality Metric for Generative Reconstruction Models](https://mlsanity.com/paper/2407.12780/) (2024)
- [Pathological Semantics-Preserving Learning for H&E-to-IHC Virtual Staining](https://mlsanity.com/paper/2407.03655/) (2024)
- [Vector Quantized Image-to-Image Translation](https://mlsanity.com/paper/2207.13286/) (2022)
- [Predicting Ki67, ER, PR, and HER2 Statuses from H&E-stained Breast Cancer Images](https://mlsanity.com/paper/2308.01982/) (2023)
- [DeepLIIF: An Online Platform for Quantification of Clinical Pathology Slides](https://mlsanity.com/paper/2204.04494/) (2022)
- [DeeperHistReg: Robust Whole Slide Images Registration Framework](https://mlsanity.com/paper/2404.14434/) (2024)
- [HistDiST: Histopathological Diffusion-based Stain Transfer](https://mlsanity.com/paper/2505.06793/) (2025)
- [Virtual Full-stack Scanning of Brain MRI via Imputing Any Quantised Code](https://mlsanity.com/paper/2501.18328/) (2025)

## Abstract (arXiv)

Virtual immunohistochemistry (IHC) aims to computationally synthesize molecular staining patterns from routine Hematoxylin and Eosin (H&E) images, offering a cost-effective and tissue-efficient alternative to traditional physical staining. However, this task is particularly challenging: H&E morphology provides ambiguous cues about protein expression, and similar tissue structures may correspond to distinct molecular states. Most existing methods focus on direct appearance synthesis to implicitly achieve cross-modal generation, often resulting in semantic inconsistencies due to insufficient structural priors. In this paper, we propose Pathology-Aware Integrated Next-Scale Transformation (PAINT), a visual autoregressive framework that reformulates the synthesis process as a structure-first conditional generation task. Unlike direct image translation, PAINT enforces a causal order by resolving molecular details conditioned on a global structural layout. Central to this approach is the introduction of a Spatial Structural Start Map (3S-Map), which grounds the autoregressive initialization in observed morphology, ensuring deterministic, spatially aligned synthesis. Experiments on the IHC4BC and MIST datasets demonstrate that PAINT outperforms state-of-the-art methods in structural fidelity and clinical downstream tasks, validating the potential of structure-guided autoregressive modeling.