# Streamline pathology foundation model by cross-magnification distillation

arXiv: https://arxiv.org/abs/2509.23097 · HTML: https://mlsanity.com/paper/2509.23097/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 18 of 18)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining](https://mlsanity.com/paper/2210.10341/) (2022)
- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [Virchow2: Scaling Self-Supervised Mixed Magnification Models in Pathology](https://mlsanity.com/paper/2408.00738/) (2024)
- [Artificial Intelligence for Digital and Computational Pathology](https://mlsanity.com/paper/2401.06148/) (2023)
- [Towards A Generalizable Pathology Foundation Model via Unified Knowledge Distillation](https://mlsanity.com/paper/2407.18449/) (2024)
- [Development and Evaluation of a Deep Neural Network for Histologic Classification of Renal Cell Carcinoma on Biopsy and Surgical Resection Slides](https://mlsanity.com/paper/2010.16380/) (2020)
- [LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settings](https://mlsanity.com/paper/2503.19740/) (2025)
- [PrPSeg: Universal Proposition Learning for Panoramic Renal Pathology Segmentation](https://mlsanity.com/paper/2402.19286/) (2024)
- [SPIDER: A Comprehensive Multi-Organ Supervised Pathology Dataset and Baseline Models](https://mlsanity.com/paper/2503.02876/) (2025)
- [A Simple Recipe for Competitive Low-compute Self supervised Vision Models](https://mlsanity.com/paper/2301.09451/) (2023)
- [Computational Pathology for Accurate Prediction of Breast Cancer Recurrence: Development and Validation of a Deep Learning-based Tool](https://mlsanity.com/paper/2409.15491/) (2024)
- [Cross-attention-based saliency inference for predicting cancer metastasis on whole slide images](https://mlsanity.com/paper/2309.09412/) (2023)

## Built on by (most cited first, 5 of 5)

- [MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding](https://mlsanity.com/paper/2512.06581/) (2025)
- [Can LLM-Generated Text Empower Surgical Vision-Language Pre-training?](https://mlsanity.com/paper/2604.18134/) (2026)
- [M3Net: A Macro-to-Meso-to-Micro Clinical-inspired Hierarchical 3D Network for Pulmonary Nodule Classification](https://mlsanity.com/paper/2605.12570/) (2026)
- [Unified Multi-Foundation-Model Slide Representation for Pan-Cancer Recognition and Text-Guided Tumor Localization](https://mlsanity.com/paper/2604.22846/) (2026)
- [What Makes High-Magnification Knowledge Transferable? A Study of Cross-Resolution Distillation in Whole-Slide Imaging](https://mlsanity.com/paper/2609.36407/) (2026)

## Abstract (arXiv)

Foundation models (FM) have transformed computational pathology but remain computationally prohibitive for clinical deployment due to their massive parameter counts and high-magnification processing requirements. Here, we introduce XMAG, a lightweight FM developed through corss-magnification distillation that transfers knowledge from state-of-the-art 20x magnification teacher to an efficient 5x magnification student architecture. XMAG employs a compact backbone and operates entirely at 5x, requiring 11.3 times fewer patches per whole slide image (WSI) compared to existing approaches. Our Novel distillation framework incorporates dual-level knowledge transfer, aligning both global image representations and local spatial token mapping. We trained XMAG on 3.49 million images curated from publicly available datasets and evaluated performance across six clinically relevant histopathology analysis tasks spanning multiple cancer types. XMAG achieved diagnostic accuracy within 1% of substantially larger foundation models while delivering 30-fold processing acceleration, reaching 8.8 WSIs per minute processing speed. Our cross-institutional validation confirmed robust generalization. Further, we developed an end-to-end training strategy to further boost our model's performance to approach the larger FMs' performance. These results establish cross-magnification distillation as a viable approach for deploying FM capabilities in resource-constrained clinical environments, potentially enabling real-time pathology AI integration.