# Label-free Concept Based Multiple Instance Learning for Gigapixel Histopathology

arXiv: https://arxiv.org/abs/2501.02922 · HTML: https://mlsanity.com/paper/2501.02922/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 6 Jan 2025.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Deep neural network models for computational histopathology: A survey](https://mlsanity.com/paper/1912.12378/) (2019)
- [Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery](https://mlsanity.com/paper/2407.14499/) (2024)
- [B-cos Networks: Alignment is All We Need for Interpretability](https://mlsanity.com/paper/2205.10268/) (2022)
- [Differentiable Patch Selection for Image Recognition](https://mlsanity.com/paper/2104.03059/) (2021)
- [Additive MIL: Intrinsically Interpretable Multiple Instance Learning for Pathology](https://mlsanity.com/paper/2206.01794/) (2022)
- [Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images](https://mlsanity.com/paper/2204.12454/) (2022)
- [A Multi-resolution Model for Histopathology Image Classification and Localization with Multiple Instance Learning](https://mlsanity.com/paper/2011.02679/) (2020)
- [Regression Concept Vectors for Bidirectional Explanations in Histopathology](https://mlsanity.com/paper/1904.04520/) (2019)
- [SI-MIL: Taming Deep MIL for Self-Interpretability in Gigapixel Histopathology](https://mlsanity.com/paper/2312.15010/) (2023)
- [MProtoNet: A Case-Based Interpretable Model for Brain Tumor Classification with 3D Multi-parametric Magnetic Resonance Imaging](https://mlsanity.com/paper/2304.06258/) (2023)
- [Inherently Interpretable Multi-Label Classification Using Class-Specific Counterfactuals](https://mlsanity.com/paper/2303.00500/) (2023)
- [Improving Interpretability for Computer-aided Diagnosis tools on Whole Slide Imaging with Multiple Instance Learning and Gradient-based Explanations](https://mlsanity.com/paper/2009.14001/) (2020)

## Built on by (most cited first, 8 of 8)

- [GECKO: Gigapixel Vision-Concept Contrastive Pretraining in Histopathology](https://mlsanity.com/paper/2504.01009/) (2025)
- [Contrastive Integrated Gradients: A Feature Attribution-Based Method for Explaining Whole Slide Image Classification](https://mlsanity.com/paper/2511.08464/) (2025)
- [Prototype-Based Multiple Instance Learning for Gigapixel Whole Slide Image Classification](https://mlsanity.com/paper/2503.08384/) (2025)
- [Libra-MIL: Multimodal Prototypes Stereoscopic Infused with Task-specific Language Priors for Few-shot Whole Slide Image Classification](https://mlsanity.com/paper/2511.07941/) (2025)
- [PA-MIL: Phenotype-Aware Multiple Instance Learning Guided by Language Prompting and Genotype-to-Phenotype Relationships](https://mlsanity.com/paper/2602.02558/) (2026)
- [Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology](https://mlsanity.com/paper/2606.06224/) (2026)
- [SAGE: Semantic Explainability of Attention-Based Survival Models in Computational Pathology](https://mlsanity.com/paper/2608.02803/) (2026)
- [Statistical Testing for Multiple Instance Learning via Selective Inference with Applications to Computational Pathology](https://mlsanity.com/paper/2609.31712/) (2026)

## Abstract (arXiv)

Multiple Instance Learning (MIL) methods allow for gigapixel Whole-Slide Image (WSI) analysis with only slide-level annotations. Interpretability is crucial for safely deploying such algorithms in high-stakes medical domains. Traditional MIL methods offer explanations by highlighting salient regions. However, such spatial heatmaps provide limited insights for end users. To address this, we propose a novel inherently interpretable WSI-classification approach that uses human-understandable pathology concepts to generate explanations. Our proposed Concept MIL model leverages recent advances in vision-language models to directly predict pathology concepts based on image features. The model's predictions are obtained through a linear combination of the concepts identified on the top-K patches of a WSI, enabling inherent explanations by tracing each concept's influence on the prediction. In contrast to traditional concept-based interpretable models, our approach eliminates the need for costly human annotations by leveraging the vision-language model. We validate our method on two widely used pathology datasets: Camelyon16 and PANDA. On both datasets, Concept MIL achieves AUC and accuracy scores over 0.9, putting it on par with state-of-the-art models. We further find that 87.1% (Camelyon16) and 85.3% (PANDA) of the top 20 patches fall within the tumor region. A user study shows that the concepts identified by our model align with the concepts used by pathologists, making it a promising strategy for human-interpretable WSI classification.