# Phikon-v2, A large and public feature extractor for biomarker prediction

arXiv: https://arxiv.org/abs/2409.09173 · HTML: https://mlsanity.com/paper/2409.09173/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 63 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [Artificial Intelligence for Digital and Computational Pathology](https://mlsanity.com/paper/2401.06148/) (2023)
- [A Clinical Benchmark of Public Self-Supervised Pathology Foundation Models](https://mlsanity.com/paper/2407.06508/) (2024)
- [Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology](https://mlsanity.com/paper/2203.00585/) (2022)
- [Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology](https://mlsanity.com/paper/2012.03583/) (2020)
- [Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides](https://mlsanity.com/paper/2112.02222/) (2021)
- [Computational Pathology at Health System Scale -- Self-Supervised Foundation Models from Three Billion Images](https://mlsanity.com/paper/2310.07033/) (2023)
- [Towards A Generalizable Pathology Foundation Model via Unified Knowledge Distillation](https://mlsanity.com/paper/2407.18449/) (2024)
- [DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer](https://mlsanity.com/paper/2107.09405/) (2021)
- [Development and Evaluation of a Deep Neural Network for Histologic Classification of Renal Cell Carcinoma on Biopsy and Surgical Resection Slides](https://mlsanity.com/paper/2010.16380/) (2020)
- [Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications](https://mlsanity.com/paper/2112.05760/) (2021)
- [Domain-specific optimization and diverse evaluation of self-supervised models for histopathology](https://mlsanity.com/paper/2310.13259/) (2023)
- [Pre-training with Random Orthogonal Projection Image Modeling](https://mlsanity.com/paper/2310.18737/) (2023)
- [HEROHE Challenge: assessing HER2 status in breast cancer without immunohistochemistry or in situ hybridization](https://mlsanity.com/paper/2111.04738/) (2021)
- [Screen Them All: High-Throughput Pan-Cancer Genetic and Phenotypic Biomarker Screening from H&E Whole Slide Images](https://mlsanity.com/paper/2408.09554/) (2024)
- [On Good Practices for Task-Specific Distillation of Large Pretrained Visual Models](https://mlsanity.com/paper/2402.11305/) (2024)

## Built on by (most cited first, 30 of 74)

- [LLM Agents Making Agent Tools](https://mlsanity.com/paper/2502.11705/) (2025)
- [Distilling foundation models for robust and efficient models in digital pathology](https://mlsanity.com/paper/2501.16239/) (2025)
- [PathBench: A comprehensive comparison benchmark for pathology foundation models towards precision oncology](https://mlsanity.com/paper/2505.20202/) (2025)
- [A Survey on Computational Pathology Foundation Models: Datasets, Adaptation Strategies, and Evaluation Tasks](https://mlsanity.com/paper/2501.15724/) (2025)
- [A Survey of Pathology Foundation Model: Progress and Future Directions](https://mlsanity.com/paper/2504.04045/) (2025)
- [Atlas: A Novel Pathology Foundation Model by Mayo Clinic, Charit'e, and Aignostics](https://mlsanity.com/paper/2501.05409/) (2025)
- [Evaluating Computational Pathology Foundation Models for Prostate Cancer Grading under Distribution Shifts](https://mlsanity.com/paper/2410.06723/) (2024)
- [Scanner-Induced Domain Shifts Undermine the Robustness of Pathology Foundation Models](https://mlsanity.com/paper/2601.04163/) (2026)
- [SurGen: 1020 H&E-stained Whole Slide Images With Survival and Genetic Markers](https://mlsanity.com/paper/2502.04946/) (2025)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [Atlas 2 -- Foundation models for clinical deployment](https://mlsanity.com/paper/2601.05148/) (2026)
- [MTBBench: A Multimodal Sequential Clinical Decision-Making Benchmark in Oncology](https://mlsanity.com/paper/2511.20490/) (2025)
- [Pathryoshka: Compressing Pathology Foundation Models via Multi-Teacher Knowledge Distillation with Nested Embeddings](https://mlsanity.com/paper/2511.23204/) (2025)
- [TICON: A Slide-Level Tile Contextualizer for Histopathology Representation Learning](https://mlsanity.com/paper/2512.21331/) (2025)
- [Gen-SIS: Generative Self-augmentation Improves Self-supervised Learning](https://mlsanity.com/paper/2412.01672/) (2024)
- [Going Beyond H&E and Oncology: How Do Histopathology Foundation Models Perform for Multi-stain IHC and Immunology?](https://mlsanity.com/paper/2410.21560/) (2024)
- [Pretext Matters: An Empirical Study of SSL Methods in Medical Imaging](https://mlsanity.com/paper/2603.22649/) (2026)
- [MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding](https://mlsanity.com/paper/2603.23067/) (2026)
- [MOOZY: A Patient-First Foundation Model for Computational Pathology](https://mlsanity.com/paper/2603.27048/) (2026)
- [Benchmarking Foundation Models for Mitotic Figure Classification](https://mlsanity.com/paper/2508.04441/) (2025)
- [Controllable Latent Space Augmentation for Digital Pathology](https://mlsanity.com/paper/2508.14588/) (2025)
- [Learning the Language of Histopathology Images reveals Prognostic Subgroups in Invasive Lung Adenocarcinoma Patients](https://mlsanity.com/paper/2508.16742/) (2025)
- [Towards Cellular-Scale Interpretability in Pathology Foundation Models for Biomarker Assessment](https://mlsanity.com/paper/2511.05150/) (2025)
- [Uni-Hema: Unified Model for Digital Hematopathology](https://mlsanity.com/paper/2511.13889/) (2025)
- [Towards Autonomous and Auditable Medical Imaging Model Development](https://mlsanity.com/paper/2607.10522/) (2026)
- [Unifying Multiple Foundation Models for Advanced Computational Pathology](https://mlsanity.com/paper/2503.00736/) (2025)
- [PhenoBench: A Comprehensive Benchmark for Cell Phenotyping](https://mlsanity.com/paper/2507.03532/) (2025)
- [Leveraging Pathology Foundation Models for Panoptic Segmentation of Melanoma in H&E Images](https://mlsanity.com/paper/2507.13974/) (2025)
- [Morphology-Aware Prognostic model for Five-Year Survival Prediction in Colorectal Cancer from H&E Whole Slide Images](https://mlsanity.com/paper/2510.14800/) (2025)
- [TomoGraphView: 3D Medical Image Classification with Omnidirectional Slice Representations and Graph Neural Networks](https://mlsanity.com/paper/2511.09605/) (2025)

## Abstract (arXiv)

Gathering histopathology slides from over 100 publicly available cohorts, we compile a diverse dataset of 460 million pathology tiles covering more than 30 cancer sites. Using this dataset, we train a large self-supervised vision transformer using DINOv2 and publicly release one iteration of this model for further experimentation, coined Phikon-v2. While trained on publicly available histology slides, Phikon-v2 surpasses our previously released model (Phikon) and performs on par with other histopathology foundation models (FM) trained on proprietary data. Our benchmarks include eight slide-level tasks with results reported on external validation cohorts avoiding any data contamination between pre-training and evaluation datasets. Our downstream training procedure follows a simple yet robust ensembling strategy yielding a +1.75 AUC increase across tasks and models compared to one-shot retraining (p<0.001). We compare Phikon (ViT-B) and Phikon-v2 (ViT-L) against 14 different histology feature extractors, making our evaluation the most comprehensive to date. Our result support evidences that DINOv2 handles joint model and data scaling better than iBOT. Also, we show that recent scaling efforts are overall beneficial to downstream performance in the context of biomarker prediction with GigaPath and H-Optimus-0 (two ViT-g with 1.1B parameters each) standing out. However, the statistical margins between the latest top-performing FMs remain mostly non-significant; some even underperform on specific indications or tasks such as MSI prediction - deposed by a 13x smaller model developed internally. While latest foundation models may exhibit limitations for clinical deployment, they nonetheless offer excellent grounds for the development of more specialized and cost-efficient histology encoders fueling AI-guided diagnostic tools.