# Harnessing Adversarial Distillation to Customise Debiased, Disease-Specific Pathology Foundation Models for Breast Cancer

arXiv: https://arxiv.org/abs/2608.01356 · HTML: https://mlsanity.com/paper/2608.01356/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 2 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Eigen-CAM: Class Activation Map using Principal Components](https://mlsanity.com/paper/2008.00299/) (2020)
- [Virchow2: Scaling Self-Supervised Mixed Magnification Models in Pathology](https://mlsanity.com/paper/2408.00738/) (2024)
- [BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images](https://mlsanity.com/paper/2111.04740/) (2021)
- [Phikon-v2, A large and public feature extractor for biomarker prediction](https://mlsanity.com/paper/2409.09173/) (2024)
- [Accelerating Data Processing and Benchmarking of AI Models for Pathology](https://mlsanity.com/paper/2502.06750/) (2025)
- [A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Model](https://mlsanity.com/paper/2407.15362/) (2024)
- [Distilling foundation models for robust and efficient models in digital pathology](https://mlsanity.com/paper/2501.16239/) (2025)
- [Domain Generalization in Computational Pathology: Survey and Guidelines](https://mlsanity.com/paper/2310.19656/) (2023)
- [Do Histopathological Foundation Models Eliminate Batch Effects? A Comparative Study](https://mlsanity.com/paper/2411.05489/) (2024)
- [A Survey on Computational Pathology Foundation Models: Datasets, Adaptation Strategies, and Evaluation Tasks](https://mlsanity.com/paper/2501.15724/) (2025)
- [A Survey of Pathology Foundation Model: Progress and Future Directions](https://mlsanity.com/paper/2504.04045/) (2025)

## Abstract (arXiv)

Pathology foundation models (PFMs) provide strong tissue representations and have become central to digital pathology. However, deployment in disease-specific settings is limited by 1) the high computational cost of billion-parameter PFMs and 2) distribution mismatch and non-biological bias inherited from pan-cancer, multi-centre pre-training, including site-specific signatures and imbalanced disease prevalence. These factors can encourage shortcut learning and under-emphasise subtle morphology required for reliable modelling of a specific cancer type. We present SmartStu (a Smart Student), a framework to customise compact, breast-cancer-specific PFMs via distillation whilst mitigating confounding. SmartStu distils representations from multiple teacher PFMs into a lightweight student backbone. Crucially, we introduce adversarial distillation that leverages a dedicated noise model trained to predict nuisance, edge-dominated cues on the distillation set. Using this noise model as a counterexample, the adversarial objective encourages the student to recognise, yet suppress, features predictive of nuisance targets. We further incorporate multi-teacher ensemble distillation and an auxiliary self-supervised objective with artefact injection. We validate SmartStu on three external cohorts (Yale HER2, SLN-Breast, and BRACS) with multiple tiny backbones. SmartStu yields breast-cancer-specific PFMs that are over 30× smaller than general PFMs whilst largely preserving, and sometimes improving, downstream performance measured by balanced accuracy (bAcc) and AUC. Code is available at https://github.com/zwchen03/advDistall.