# PTCMIL: Multiple Instance Learning via Prompt Token Clustering for Whole Slide Image Analysis

arXiv: https://arxiv.org/abs/2507.18848 · HTML: https://mlsanity.com/paper/2507.18848/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Builds on (most cited first, 30 of 31)

- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [Visual Prompting via Image Inpainting](https://mlsanity.com/paper/2209.00647/) (2022)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images](https://mlsanity.com/paper/2111.04740/) (2021)
- [Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks](https://mlsanity.com/paper/2107.13048/) (2021)
- [Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformer](https://mlsanity.com/paper/2204.08680/) (2022)
- [Brain Network Transformer](https://mlsanity.com/paper/2210.06681/) (2022)
- [MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology](https://mlsanity.com/paper/2403.06800/) (2024)
- [Understanding and Improving Visual Prompting: A Label-Mapping Perspective](https://mlsanity.com/paper/2211.11635/) (2022)
- [Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Prediction](https://mlsanity.com/paper/2306.08330/) (2023)
- [A graph-transformer for whole slide image classification](https://mlsanity.com/paper/2205.09671/) (2022)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology](https://mlsanity.com/paper/2405.11643/) (2024)
- [Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2103.10626/) (2021)
- [Histopathology Whole Slide Image Analysis with Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2307.04189/) (2023)
- [Handcrafted Histological Transformer (H2T): Unsupervised Representation of Whole Slide Images](https://mlsanity.com/paper/2202.07001/) (2022)
- [ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2502.08391/) (2025)
- [DGR-MIL: Exploring Diverse Global Representation in Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2407.03575/) (2024)
- [Agglomerative Token Clustering](https://mlsanity.com/paper/2409.11923/) (2024)
- [MulGT: Multi-task Graph-Transformer with Task-aware Knowledge Injection and Domain Knowledge-driven Pooling for Whole Slide Image Analysis](https://mlsanity.com/paper/2302.10574/) (2023)
- [Iteratively Coupled Multiple Instance Learning from Instance to Bag Classifier for Whole Slide Image Classification](https://mlsanity.com/paper/2303.15749/) (2023)

## Built on by (most cited first, 3 of 3)

- [CGRL: Concept-Guided Pruning and Representation Learning for Whole-Slide Image Classification](https://mlsanity.com/paper/2607.12556/) (2026)
- [A Multi-Cohort Validation of Censoring-Aware Conformal Lower Predictive Bounds for Pathology Survival Models](https://mlsanity.com/paper/2608.04025/) (2026)
- [One Tile, Multiple Instances: Rethinking MIL for Sparse Diagnostic Evidence](https://mlsanity.com/paper/2610.04853/) (2026)

## Abstract (arXiv)

Multiple Instance Learning (MIL) has advanced WSI analysis but struggles with the complexity and heterogeneity of WSIs. Existing MIL methods face challenges in aggregating diverse patch information into robust WSI representations. While ViTs and clustering-based approaches show promise, they are computationally intensive and fail to capture task-specific and slide-specific variability. To address these limitations, we propose PTCMIL, a novel Prompt Token Clustering-based ViT for MIL aggregation. By introducing learnable prompt tokens into the ViT backbone, PTCMIL unifies clustering and prediction tasks in an end-to-end manner. It dynamically aligns clustering with downstream tasks, using projection-based clustering tailored to each WSI, reducing complexity while preserving patch heterogeneity. Through token merging and prototype-based pooling, PTCMIL efficiently captures task-relevant patterns. Extensive experiments on eight datasets demonstrate its superior performance in classification and survival analysis tasks, outperforming state-of-the-art methods. Systematic ablation studies confirm its robustness and strong interpretability. The code is released at https://github.com/ubc-tea/PTCMIL.