# Validation of Whole-Slide Foundation Models for Image Retrieval in TCGA Data

arXiv: https://arxiv.org/abs/2605.00902 · HTML: https://mlsanity.com/paper/2605.00902/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 28 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Multimodal Whole Slide Foundation Model for Pathology](https://mlsanity.com/paper/2411.19666/) (2024)
- [PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology](https://mlsanity.com/paper/2405.10254/) (2024)
- [A Clinical Benchmark of Public Self-Supervised Pathology Foundation Models](https://mlsanity.com/paper/2407.06508/) (2024)
- [Accelerating Data Processing and Benchmarking of AI Models for Pathology](https://mlsanity.com/paper/2502.06750/) (2025)
- [A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Model](https://mlsanity.com/paper/2407.15362/) (2024)
- [Benchmarking foundation models as feature extractors for weakly-supervised computational pathology](https://mlsanity.com/paper/2408.15823/) (2024)
- [Yottixel -- An Image Search Engine for Large Archives of Histopathology Whole Slide Images](https://mlsanity.com/paper/1911.08748/) (2019)
- [Multistain Pretraining for Slide Representation Learning in Pathology](https://mlsanity.com/paper/2408.02859/) (2024)
- [Pan-Cancer Diagnostic Consensus Through Searching Archival Histopathology Images Using Artificial Intelligence](https://mlsanity.com/paper/1911.08736/) (2019)
- [A Survey of Pathology Foundation Model: Progress and Future Directions](https://mlsanity.com/paper/2504.04045/) (2025)
- [On Image Search in Histopathology](https://mlsanity.com/paper/2401.08699/) (2024)
- [MinMax Radon Barcodes for Medical Image Retrieval](https://mlsanity.com/paper/1610.00318/) (2016)
- [Analysis and Validation of Image Search Engines in Histopathology](https://mlsanity.com/paper/2401.03271/) (2024)
- [Beyond the Failures: Rethinking Foundation Models in Pathology](https://mlsanity.com/paper/2510.23807/) (2025)

## Abstract (arXiv)

Foundation models are reshaping computational histopathology, yet their value for whole-slide image retrieval relative to strong patch-based and supervised aggregation baselines remains unclear. We benchmarked ten pipelines on 9,387 diagnostic slides spanning 17 organs and 60 diagnoses from The Cancer Genome Atlas (TCGA) using patient-level leave-one-patient-out evaluation. Methods included four pre-trained slide foundation models, a supervised attention-based multiple instance learning (ABMIL) aggregator on patch embeddings, and patch-level retrieval across five sampling densities. Performance varied more across organs and diagnoses than across architectures. Although the slide foundation model TITAN achieved the strongest overall results, its advantage was modest; ABMIL and patch-based methods reached comparable Top-1 and Top-3 accuracy, with no model consistently dominant. Morphologically distinctive entities approached ceiling performance, while rare, heterogeneous, and closely related subtypes remained challenging. Misclassifications aligned with organs exhibiting known inter-observer variability, suggesting an intrinsic ceiling for morphology-only retrieval. Performance was driven primarily by patch-level feature representations, with limited benefit from slide-level aggregation, indicating aggregation may be unnecessary in many settings. These findings argue against a universally optimal architecture and instead support organ-resolved benchmarking, diagnosis-aware or ensemble strategies, stronger feature representations, and multimodal retrieval frameworks. Notably, even the best model achieved only ≈ 68% ± 21% retrieval accuracy on TCGA, and some subtypes showed 0% accuracy across all methods, highlighting fundamental limitations of morphology-based representations and the need for substantial progress before reliable clinical deployment.