# Pathology-knowledge Enhanced Multi-instance Prompt Learning for Few-shot Whole Slide Image Classification

arXiv: https://arxiv.org/abs/2407.10814 · HTML: https://mlsanity.com/paper/2407.10814/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 43 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Visual-Language Prompt Tuning with Knowledge-guided Context Optimization](https://mlsanity.com/paper/2303.13283/) (2023)
- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [Quilt-1M: One Million Image-Text Pairs for Histopathology](https://mlsanity.com/paper/2306.11207/) (2023)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis](https://mlsanity.com/paper/1912.08937/) (2019)
- [Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images](https://mlsanity.com/paper/2306.07831/) (2023)
- [Artificial Intelligence for Digital and Computational Pathology](https://mlsanity.com/paper/2401.06148/) (2023)
- [Interventional Bag Multi-Instance Learning On Whole-Slide Pathological Images](https://mlsanity.com/paper/2303.06873/) (2023)
- [Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting](https://mlsanity.com/paper/2304.03307/) (2023)
- [Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images](https://mlsanity.com/paper/2001.01599/) (2020)
- [CAMEL: A Weakly Supervised Learning Framework for Histopathology Image Segmentation](https://mlsanity.com/paper/1908.10555/) (2019)
- [Task-specific Fine-tuning via Variational Information Bottleneck for Weakly-supervised Pathology Whole Slide Image Classification](https://mlsanity.com/paper/2303.08446/) (2023)
- [Histopathology Whole Slide Image Analysis with Heterogeneous Graph Representation Learning](https://mlsanity.com/paper/2307.04189/) (2023)
- [Text-guided Foundation Model Adaptation for Pathological Image Classification](https://mlsanity.com/paper/2307.14901/) (2023)
- [The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification](https://mlsanity.com/paper/2305.17891/) (2023)
- [DGMIL: Distribution Guided Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2206.08861/) (2022)
- [Towards Label-efficient Automatic Diagnosis and Analysis: A Comprehensive Survey of Advanced Deep Learning-based Weakly-supervised, Semi-supervised and Self-supervised Techniques in Histopathological Image Analysis](https://mlsanity.com/paper/2208.08789/) (2022)
- [Semantic Prompt for Few-Shot Image Recognition](https://mlsanity.com/paper/2303.14123/) (2023)
- [Deep Weakly-Supervised Learning Methods for Classification and Localization in Histology Images: A Survey](https://mlsanity.com/paper/1909.03354/) (2019)
- [ConSlide: Asynchronous Hierarchical Interaction Transformer with Breakup-Reorganize Rehearsal for Continual Whole Slide Image Analysis](https://mlsanity.com/paper/2308.13324/) (2023)

## Built on by (most cited first, 13 of 13)

- [FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification](https://mlsanity.com/paper/2411.14743/) (2024)
- [A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Models](https://mlsanity.com/paper/2408.14496/) (2024)
- [MGPATH: Vision-Language Model with Multi-Granular Prompt Learning for Few-Shot WSI Classification](https://mlsanity.com/paper/2502.07409/) (2025)
- [Few-Shot Learning from Gigapixel Images via Hierarchical Vision-Language Alignment and Modeling](https://mlsanity.com/paper/2505.17982/) (2025)
- [HiLa: Hierarchical Vision-Language Collaboration for Cancer Survival Prediction](https://mlsanity.com/paper/2507.04613/) (2025)
- [Multimodal Prototype Alignment for Semi-supervised Pathology Image Segmentation](https://mlsanity.com/paper/2508.19574/) (2025)
- [Exploiting Low-Dimensional Manifold of Features for Few-Shot Whole Slide Image Classification](https://mlsanity.com/paper/2505.15504/) (2025)
- [MUSE: Harnessing Precise and Diverse Semantics for Few-Shot Whole Slide Image Classification](https://mlsanity.com/paper/2602.20873/) (2026)
- [Efficient Whole Slide Image Classification through Fisher Vector Representation](https://mlsanity.com/paper/2411.08530/) (2024)
- [Weakly Semi-supervised Whole Slide Image Classification by Two-level Cross Consistency Supervision](https://mlsanity.com/paper/2504.12132/) (2025)
- [MOC: Meta-Optimized Classifier for Few-Shot Whole Slide Image Classification](https://mlsanity.com/paper/2508.09967/) (2025)
- [ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets](https://mlsanity.com/paper/2602.19708/) (2026)
- [Parameter-efficient Prompt Tuning and Hierarchical Textual Guidance for Few-shot Whole Slide Image Classification](https://mlsanity.com/paper/2603.21504/) (2026)

## Abstract (arXiv)

Current multi-instance learning algorithms for pathology image analysis often require a substantial number of Whole Slide Images for effective training but exhibit suboptimal performance in scenarios with limited learning data. In clinical settings, restricted access to pathology slides is inevitable due to patient privacy concerns and the prevalence of rare or emerging diseases. The emergence of the Few-shot Weakly Supervised WSI Classification accommodates the significant challenge of the limited slide data and sparse slide-level labels for diagnosis. Prompt learning based on the pre-trained models (eg, CLIP) appears to be a promising scheme for this setting; however, current research in this area is limited, and existing algorithms often focus solely on patch-level prompts or confine themselves to language prompts. This paper proposes a multi-instance prompt learning framework enhanced with pathology knowledge, ie, integrating visual and textual prior knowledge into prompts at both patch and slide levels. The training process employs a combination of static and learnable prompts, effectively guiding the activation of pre-trained models and further facilitating the diagnosis of key pathology patterns. Lightweight Messenger (self-attention) and Summary (attention-pooling) layers are introduced to model relationships between patches and slides within the same patient data. Additionally, alignment-wise contrastive losses ensure the feature-level alignment between visual and textual learnable prompts for both patches and slides. Our method demonstrates superior performance in three challenging clinical tasks, significantly outperforming comparative few-shot methods.