# RudolfV: A Foundation Model by Pathologists for Pathologists

arXiv: https://arxiv.org/abs/2401.04079 · HTML: https://mlsanity.com/paper/2401.04079/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 72 references that are in the map.
- First posted on arXiv 8 Jan 2024.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [SimpleShot: Revisiting Nearest-Neighbor Classification for Few-Shot Learning](https://mlsanity.com/paper/1911.04623/) (2019)
- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [Self supervised contrastive learning for digital histopathology](https://mlsanity.com/paper/2011.13971/) (2020)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)
- [Virchow: A Million-Slide Digital Pathology Foundation Model](https://mlsanity.com/paper/2309.07778/) (2023)
- [Divide and Contrast: Self-supervised Learning from Uncurated Data](https://mlsanity.com/paper/2105.08054/) (2021)
- [A Petri Dish for Histopathology Image Analysis](https://mlsanity.com/paper/2101.12355/) (2021)
- [Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology](https://mlsanity.com/paper/2203.00585/) (2022)
- [A General-Purpose Self-Supervised Model for Computational Pathology](https://mlsanity.com/paper/2308.15474/) (2023)
- [Yottixel -- An Image Search Engine for Large Archives of Histopathology Whole Slide Images](https://mlsanity.com/paper/1911.08748/) (2019)

## Built on by (most cited first, 30 of 41)

- [Multimodal Whole Slide Foundation Model for Pathology](https://mlsanity.com/paper/2411.19666/) (2024)
- [Virchow2: Scaling Self-Supervised Mixed Magnification Models in Pathology](https://mlsanity.com/paper/2408.00738/) (2024)
- [PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology](https://mlsanity.com/paper/2405.10254/) (2024)
- [A Clinical Benchmark of Public Self-Supervised Pathology Foundation Models](https://mlsanity.com/paper/2407.06508/) (2024)
- [Distilling foundation models for robust and efficient models in digital pathology](https://mlsanity.com/paper/2501.16239/) (2025)
- [The Clever Hans Effect in Unsupervised Learning](https://mlsanity.com/paper/2408.08041/) (2024)
- [Benchmarking Pathology Foundation Models: Adaptation Strategies and Scenarios](https://mlsanity.com/paper/2410.16038/) (2024)
- [xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology](https://mlsanity.com/paper/2406.04280/) (2024)
- [A Survey on Computational Pathology Foundation Models: Datasets, Adaptation Strategies, and Evaluation Tasks](https://mlsanity.com/paper/2501.15724/) (2025)
- [A Survey of Pathology Foundation Model: Progress and Future Directions](https://mlsanity.com/paper/2504.04045/) (2025)
- [Atlas: A Novel Pathology Foundation Model by Mayo Clinic, Charit'e, and Aignostics](https://mlsanity.com/paper/2501.05409/) (2025)
- [A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Models](https://mlsanity.com/paper/2408.14496/) (2024)
- [SurGen: 1020 H&E-stained Whole Slide Images With Survival and Genetic Markers](https://mlsanity.com/paper/2502.04946/) (2025)
- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [Pathology Foundation Models](https://mlsanity.com/paper/2407.21317/) (2024)
- [EXAONEPath 1.0 Patch-level Foundation Model for Pathology](https://mlsanity.com/paper/2408.00380/) (2024)
- [MeDi: Metadata-Guided Diffusion Models for Mitigating Biases in Tumor Classification](https://mlsanity.com/paper/2506.17140/) (2025)
- [Atlas 2 -- Foundation models for clinical deployment](https://mlsanity.com/paper/2601.05148/) (2026)
- [From Classical Machine Learning to Emerging Foundation Models: Review on Multimodal Data Integration for Cancer Research](https://mlsanity.com/paper/2507.09028/) (2025)
- [Revisiting Automatic Data Curation for Vision Foundation Models in Digital Pathology](https://mlsanity.com/paper/2503.18709/) (2025)
- [MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding](https://mlsanity.com/paper/2603.23067/) (2026)
- [DiNO-Diffusion. Scaling Medical Diffusion via Self-Supervised Pre-Training](https://mlsanity.com/paper/2407.11594/) (2024)
- [Benchmarking Foundation Models for Mitotic Figure Classification](https://mlsanity.com/paper/2508.04441/) (2025)
- [PLUTO-4: Frontier Pathology Foundation Models](https://mlsanity.com/paper/2511.02826/) (2025)
- [Foundation Models in Biomedical Imaging: Turning Hype into Reality](https://mlsanity.com/paper/2512.15808/) (2025)
- [Distilling Lightweight Domain Experts from Large ML Models by Identifying Relevant Subspaces](https://mlsanity.com/paper/2601.05913/) (2026)
- [Beyond Attention Heatmaps: How to Get Better Explanations for Multiple Instance Learning Models in Histopathology](https://mlsanity.com/paper/2603.08328/) (2026)
- [OpenTME: An Open Dataset of AI-powered H&E Tumor Microenvironment Profiles from TCGA](https://mlsanity.com/paper/2604.12075/) (2026)
- [Foundational Models for Pathology and Endoscopy Images: Application for Gastric Inflammation](https://mlsanity.com/paper/2406.18249/) (2024)

## Abstract (arXiv)

Artificial intelligence has started to transform histopathology impacting clinical diagnostics and biomedical research. However, while many computational pathology approaches have been proposed, most current AI models are limited with respect to generalization, application variety, and handling rare diseases. Recent efforts introduced self-supervised foundation models to address these challenges, yet existing approaches do not leverage pathologist knowledge by design. In this study, we present a novel approach to designing foundation models for computational pathology, incorporating pathologist expertise, semi-automated data curation, and a diverse dataset from over 15 laboratories, including 58 tissue types, and encompassing 129 different histochemical and immunohistochemical staining modalities. We demonstrate that our model "RudolfV" surpasses existing state-of-the-art foundation models across different benchmarks focused on tumor microenvironment profiling, biomarker evaluation, and reference case search while exhibiting favorable robustness properties. Our study shows how domain-specific knowledge can increase the efficiency and performance of pathology foundation models and enable novel application areas.