# Multi-Scale Prototypical Transformer for Whole Slide Image Classification

arXiv: https://arxiv.org/abs/2307.02308 · HTML: https://mlsanity.com/paper/2307.02308/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 5 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Deep neural network models for computational histopathology: A survey](https://mlsanity.com/paper/1912.12378/) (2019)
- [A graph-transformer for whole slide image classification](https://mlsanity.com/paper/2205.09671/) (2022)
- [Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images](https://mlsanity.com/paper/2001.01599/) (2020)
- [ReMix: A General and Efficient Framework for Multiple Instance Learning based Whole Slide Image Classification](https://mlsanity.com/paper/2207.01805/) (2022)

## Built on by (most cited first, 9 of 9)

- [AEM: Attention Entropy Maximization for Multiple Instance Learning based Whole Slide Image Classification](https://mlsanity.com/paper/2406.15303/) (2024)
- [RetMIL: Retentive Multiple Instance Learning for Histopathological Whole Slide Image Classification](https://mlsanity.com/paper/2403.10858/) (2024)
- [Hard Negative Sample Mining for Whole Slide Image Classification](https://mlsanity.com/paper/2410.02212/) (2024)
- [FALFormer: Feature-aware Landmarks self-attention for Whole-slide Image Classification](https://mlsanity.com/paper/2407.07340/) (2024)
- [MMNavAgent: Multi-Magnification WSI Navigation Agent for Clinically Consistent Whole-Slide Analysis](https://mlsanity.com/paper/2603.02079/) (2026)
- [Evaluating Data Augmentation Techniques for Coffee Leaf Disease Classification](https://mlsanity.com/paper/2401.05768/) (2024)
- [EFCM: Efficient Fine-tuning on Compressed Models for deployment of large models in medical image analysis](https://mlsanity.com/paper/2409.11817/) (2024)
- [Histomorphology-Guided Prototypical Multi-Instance Learning for Breast Cancer WSI Classification](https://mlsanity.com/paper/2503.17983/) (2025)
- [Can We Go Beyond Visual Features? Neural Tissue Relation Modeling for Relational Graph Analysis in Non-Melanoma Skin Histology](https://mlsanity.com/paper/2512.06949/) (2025)

## Abstract (arXiv)

Whole slide image (WSI) classification is an essential task in computational pathology. Despite the recent advances in multiple instance learning (MIL) for WSI classification, accurate classification of WSIs remains challenging due to the extreme imbalance between the positive and negative instances in bags, and the complicated pre-processing to fuse multi-scale information of WSI. To this end, we propose a novel multi-scale prototypical Transformer (MSPT) for WSI classification, which includes a prototypical Transformer (PT) module and a multi-scale feature fusion module (MFFM). The PT is developed to reduce redundant instances in bags by integrating prototypical learning into the Transformer architecture. It substitutes all instances with cluster prototypes, which are then re-calibrated through the self-attention mechanism of the Trans-former. Thereafter, an MFFM is proposed to fuse the clustered prototypes of different scales, which employs MLP-Mixer to enhance the information communication between prototypes. The experimental results on two public WSI datasets demonstrate that the proposed MSPT outperforms all the compared algorithms, suggesting its potential applications.