# PathoDuet: Foundation Models for Pathological Slide Analysis of H&E and IHC Stains

arXiv: https://arxiv.org/abs/2312.09894 · HTML: https://mlsanity.com/paper/2312.09894/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 52 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 9 of 9)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Towards a Visual-Language Foundation Model for Computational Pathology](https://mlsanity.com/paper/2307.12914/) (2023)
- [Text-guided Foundation Model Adaptation for Pathological Image Classification](https://mlsanity.com/paper/2307.14901/) (2023)
- [Objectives Matter: Understanding the Impact of Self-Supervised Objectives on Vision Transformer Representations](https://mlsanity.com/paper/2304.13089/) (2023)
- [Large-scale pretraining on pathological images for fine-tuning of small pathological benchmarks](https://mlsanity.com/paper/2303.15693/) (2023)
- [A Dual-branch Self-supervised Representation Learning Framework for Tumour Segmentation in Whole Slide Images](https://mlsanity.com/paper/2303.11019/) (2023)
- [Context-Aware Self-Supervised Learning of Whole Slide Images](https://mlsanity.com/paper/2306.04763/) (2023)
- [Language models are good pathologists: using attention-based sequence reduction and text-pretrained transformers for efficient WSI classification](https://mlsanity.com/paper/2211.07384/) (2022)

## Built on by (most cited first, 14 of 14)

- [Multistain Pretraining for Slide Representation Learning in Pathology](https://mlsanity.com/paper/2408.02859/) (2024)
- [A Survey of Pathology Foundation Model: Progress and Future Directions](https://mlsanity.com/paper/2504.04045/) (2025)
- [PathoTune: Adapting Visual Foundation Model to Pathological Specialists](https://mlsanity.com/paper/2403.16497/) (2024)
- [SurGen: 1020 H&E-stained Whole Slide Images With Survival and Genetic Markers](https://mlsanity.com/paper/2502.04946/) (2025)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine](https://mlsanity.com/paper/2402.18028/) (2024)
- [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025)
- [Towards Generalist Intelligence in Dentistry: Vision Foundation Models for Oral and Maxillofacial Radiology](https://mlsanity.com/paper/2510.14532/) (2025)
- [Unlocking adaptive digital pathology through dynamic feature learning](https://mlsanity.com/paper/2412.20430/) (2024)
- [Efficient Self-Supervised Grading of Prostate Cancer Pathology](https://mlsanity.com/paper/2501.15520/) (2025)
- [COME: Dual Structure-Semantic Learning with Collaborative MoE for Universal Lesion Detection Across Heterogeneous Ultrasound Datasets](https://mlsanity.com/paper/2508.09886/) (2025)
- [StainNet: Scaling Self-Supervised Foundation Models on Immunohistochemistry and Special Stains for Computational Pathology](https://mlsanity.com/paper/2512.10326/) (2025)
- [When Confidence Lacks Concepts: Interpretable OOD Detection via Representation Perturbations](https://mlsanity.com/paper/2606.16196/) (2026)
- [Mitigating Batch Effects in Histopathology via Language-Mediated Robust Embedding Generation](https://mlsanity.com/paper/2606.28697/) (2026)

## Abstract (arXiv)

Large amounts of digitized histopathological data display a promising future for developing pathological foundation models via self-supervised learning methods. Foundation models pretrained with these methods serve as a good basis for downstream tasks. However, the gap between natural and histopathological images hinders the direct application of existing methods. In this work, we present PathoDuet, a series of pretrained models on histopathological images, and a new self-supervised learning framework in histopathology. The framework is featured by a newly-introduced pretext token and later task raisers to explicitly utilize certain relations between images, like multiple magnifications and multiple stains. Based on this, two pretext tasks, cross-scale positioning and cross-stain transferring, are designed to pretrain the model on Hematoxylin and Eosin (H&E) images and transfer the model to immunohistochemistry (IHC) images, respectively. To validate the efficacy of our models, we evaluate the performance over a wide variety of downstream tasks, including patch-level colorectal cancer subtyping and whole slide image (WSI)-level classification in H&E field, together with expression level prediction of IHC marker, tumor identification and slide-level qualitative analysis in IHC field. The experimental results show the superiority of our models over most tasks and the efficacy of proposed pretext tasks. The codes and models are available at https://github.com/openmedlab/PathoDuet.