# Towards A Generalizable Pathology Foundation Model via Unified Knowledge Distillation

arXiv: https://arxiv.org/abs/2407.18449 · HTML: https://mlsanity.com/paper/2407.18449/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation.
- First posted on arXiv 26 Jul 2024.

## Placed in

- Knowledge distillation / [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/)

## Built on by (most cited first, 30 of 31)

- [Phikon-v2, A large and public feature extractor for biomarker prediction](https://mlsanity.com/paper/2409.09173/) (2024)
- [Distilling foundation models for robust and efficient models in digital pathology](https://mlsanity.com/paper/2501.16239/) (2025)
- [PathBench: A comprehensive comparison benchmark for pathology foundation models towards precision oncology](https://mlsanity.com/paper/2505.20202/) (2025)
- [FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification](https://mlsanity.com/paper/2411.14743/) (2024)
- [A Survey on Computational Pathology Foundation Models: Datasets, Adaptation Strategies, and Evaluation Tasks](https://mlsanity.com/paper/2501.15724/) (2025)
- [A Survey of Pathology Foundation Model: Progress and Future Directions](https://mlsanity.com/paper/2504.04045/) (2025)
- [HMIL: Hierarchical Multi-Instance Learning for Fine-Grained Whole Slide Image Classification](https://mlsanity.com/paper/2411.07660/) (2024)
- [Large-scale Self-supervised Video Foundation Model for Intelligent Surgery](https://mlsanity.com/paper/2506.02692/) (2025)
- [A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Models](https://mlsanity.com/paper/2408.14496/) (2024)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [From Classical Machine Learning to Emerging Foundation Models: Review on Multimodal Data Integration for Cancer Research](https://mlsanity.com/paper/2507.09028/) (2025)
- [TICON: A Slide-Level Tile Contextualizer for Histopathology Representation Learning](https://mlsanity.com/paper/2512.21331/) (2025)
- [Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges](https://mlsanity.com/paper/2505.10993/) (2025)
- [Streamline pathology foundation model by cross-magnification distillation](https://mlsanity.com/paper/2509.23097/) (2025)
- [TeamPath: Building MultiModal Pathology Experts with Reasoning AI Copilots](https://mlsanity.com/paper/2511.17652/) (2025)
- [Discovering Pathology Rationale and Token Allocation for Efficient Multimodal Pathology Reasoning](https://mlsanity.com/paper/2505.15687/) (2025)
- [Benchmarking histopathology foundation models in a multi-center dataset for skin cancer subtyping](https://mlsanity.com/paper/2506.18668/) (2025)
- [A Multimodal Foundation Model to Enhance Generalizability and Data Efficiency for Pan-cancer Prognosis Prediction](https://mlsanity.com/paper/2509.12600/) (2025)
- [Foundation Models in Biomedical Imaging: Turning Hype into Reality](https://mlsanity.com/paper/2512.15808/) (2025)
- [Unifying Multiple Foundation Models for Advanced Computational Pathology](https://mlsanity.com/paper/2503.00736/) (2025)
- [Exploiting Low-Dimensional Manifold of Features for Few-Shot Whole Slide Image Classification](https://mlsanity.com/paper/2505.15504/) (2025)
- [Genome-Anchored Foundation Model Embeddings Improve Molecular Prediction from Histology Images](https://mlsanity.com/paper/2506.19681/) (2025)
- [nnMIL: A generalizable multiple instance learning framework for computational pathology](https://mlsanity.com/paper/2511.14907/) (2025)
- [Any-to-Any Learning in Computational Pathology via Triplet Multimodal Pretraining](https://mlsanity.com/paper/2505.12711/) (2025)
- [DiagR1: A Vision-Language Model Trained via Reinforcement Learning for Digestive Pathology Diagnosis](https://mlsanity.com/paper/2507.18433/) (2025)
- [Boosting Pathology Foundation Models via Few-shot Prompt-tuning for Rare Cancer Subtyping](https://mlsanity.com/paper/2508.15904/) (2025)
- [ITC-RWKV: Interactive Tissue-Cell Modeling with Recurrent Key-Value Aggregation for Histopathological Subtyping](https://mlsanity.com/paper/2510.21479/) (2025)
- [Rank-Aware Agglomeration of Foundation Models for Immunohistochemistry Image Cell Counting](https://mlsanity.com/paper/2511.12588/) (2025)
- [Fusing Pixels and Genes: Spatially-Aware Learning in Computational Pathology](https://mlsanity.com/paper/2602.13944/) (2026)
- [Federated Distillation for Whole Slide Image via Gaussian-Mixture Feature Alignment and Curriculum Integration](https://mlsanity.com/paper/2605.00578/) (2026)

## Abstract (arXiv)

Foundation models pretrained on large-scale datasets are revolutionizing the field of computational pathology (CPath). The generalization ability of foundation models is crucial for the success in various downstream clinical tasks. However, current foundation models have only been evaluated on a limited type and number of tasks, leaving their generalization ability and overall performance unclear. To address this gap, we established a most comprehensive benchmark to evaluate the performance of off-the-shelf foundation models across six distinct clinical task types, encompassing a total of 72 specific tasks, including slide-level classification, survival prediction, ROI-tissue classification, ROI retrieval, visual question answering, and report generation. Our findings reveal that existing foundation models excel at certain task types but struggle to effectively handle the full breadth of clinical tasks. To improve the generalization of pathology foundation models, we propose a unified knowledge distillation framework consisting of both expert and self-knowledge distillation, where the former allows the model to learn from the knowledge of multiple expert models, while the latter leverages self-distillation to enable image representation learning via local-global alignment. Based on this framework, we curated a dataset of 96,000 whole slide images (WSIs) and developed a Generalizable Pathology Foundation Model (GPFM). This advanced model was trained on a substantial dataset comprising 190 million images extracted from approximately 72,000 publicly available slides, encompassing 34 major tissue types. Evaluated on the established benchmark, GPFM achieves an impressive average rank of 1.6, with 42 tasks ranked 1st, while the second-best model, UNI, attains an average rank of 3.7, with only 6 tasks ranked 1st.