# Domain-specific optimization and diverse evaluation of self-supervised models for histopathology

arXiv: https://arxiv.org/abs/2310.13259 · HTML: https://mlsanity.com/paper/2310.13259/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers](https://mlsanity.com/paper/2106.10270/) (2021)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Masked Siamese Networks for Label-Efficient Learning](https://mlsanity.com/paper/2204.07141/) (2022)
- [Self-supervised Pretraining of Visual Features in the Wild](https://mlsanity.com/paper/2103.01988/) (2021)
- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [Self supervised contrastive learning for digital histopathology](https://mlsanity.com/paper/2011.13971/) (2020)
- [Intriguing Properties of Contrastive Losses](https://mlsanity.com/paper/2011.02803/) (2020)
- [On the duality between contrastive and non-contrastive self-supervised learning](https://mlsanity.com/paper/2206.02574/) (2022)
- [Virchow: A Million-Slide Digital Pathology Foundation Model](https://mlsanity.com/paper/2309.07778/) (2023)
- [Self-Path: Self-supervision for Classification of Pathology Images with Limited Annotations](https://mlsanity.com/paper/2008.05571/) (2020)
- [Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning](https://mlsanity.com/paper/2306.01859/) (2023)
- [Mitosis domain generalization in histopathology images -- The MIDOG challenge](https://mlsanity.com/paper/2204.03742/) (2022)
- [Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge](https://mlsanity.com/paper/1807.08284/) (2018)
- [Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology](https://mlsanity.com/paper/2203.00585/) (2022)
- [A General-Purpose Self-Supervised Model for Computational Pathology](https://mlsanity.com/paper/2308.15474/) (2023)
- [Development and Validation of a Deep Learning Algorithm for Improving Gleason Scoring of Prostate Cancer](https://mlsanity.com/paper/1811.06497/) (2018)
- [RandStainNA: Learning Stain-Agnostic Features from Histology Slides by Bridging Stain Augmentation and Normalization](https://mlsanity.com/paper/2206.12694/) (2022)
- [Interpretable Survival Prediction for Colorectal Cancer using Deep Learning](https://mlsanity.com/paper/2011.08965/) (2020)
- [Predicting molecular phenotypes from histopathology images: a transcriptome-wide expression-morphology analysis in breast cancer](https://mlsanity.com/paper/2009.08917/) (2020)
- [Whole-Slide Image Focus Quality: Automatic Assessment and Impact on AI Cancer Detection](https://mlsanity.com/paper/1901.04619/) (2019)
- [Multimodal Multitask Representation Learning for Pathology Biobank Metadata Prediction](https://mlsanity.com/paper/1909.07846/) (2019)
- [hist2RNA: An efficient deep learning architecture to predict gene expression from breast cancer histopathology images](https://mlsanity.com/paper/2304.04507/) (2023)

## Built on by (most cited first, 11 of 11)

- [Advancing Multimodal Medical Capabilities of Gemini](https://mlsanity.com/paper/2405.03162/) (2024)
- [Phikon-v2, A large and public feature extractor for biomarker prediction](https://mlsanity.com/paper/2409.09173/) (2024)
- [PathAlign: A vision-language model for whole slide images in histopathology](https://mlsanity.com/paper/2406.19578/) (2024)
- [A Foundational Multimodal Vision Language AI Assistant for Human Pathology](https://mlsanity.com/paper/2312.07814/) (2023)
- [A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Models](https://mlsanity.com/paper/2408.14496/) (2024)
- [A Systematic Review of Generalization Research in Medical Image Classification](https://mlsanity.com/paper/2403.12167/) (2024)
- [SurGen: 1020 H&E-stained Whole Slide Images With Survival and Genetic Markers](https://mlsanity.com/paper/2502.04946/) (2025)
- [THOR: A Versatile Foundation Model for Earth Observation Climate and Society Applications](https://mlsanity.com/paper/2601.16011/) (2026)
- [Health AI Developer Foundations](https://mlsanity.com/paper/2411.15128/) (2024)
- [Toward Scalable and Valid Conditional Independence Testing with Spectral Representations](https://mlsanity.com/paper/2512.19510/) (2025)
- [GigaPath-Flash and GigaTIME-Flash: Efficient Pathology Foundation Models for Whole-Slide and Tumor Microenvironment Analysis](https://mlsanity.com/paper/2607.18218/) (2026)

## Abstract (arXiv)

Task-specific deep learning models in histopathology offer promising opportunities for improving diagnosis, clinical research, and precision medicine. However, development of such models is often limited by availability of high-quality data. Foundation models in histopathology that learn general representations across a wide range of tissue types, diagnoses, and magnifications offer the potential to reduce the data, compute, and technical expertise necessary to develop task-specific deep learning models with the required level of model performance. In this work, we describe the development and evaluation of foundation models for histopathology via self-supervised learning (SSL). We first establish a diverse set of benchmark tasks involving 17 unique tissue types and 12 unique cancer types and spanning different optimal magnifications and task types. Next, we use this benchmark to explore and evaluate histopathology-specific SSL methods followed by further evaluation on held out patch-level and weakly supervised tasks. We found that standard SSL methods thoughtfully applied to histopathology images are performant across our benchmark tasks and that domain-specific methodological improvements can further increase performance. Our findings reinforce the value of using domain-specific SSL methods in pathology, and establish a set of high quality foundation models to enable further research across diverse applications.