# Towards Hierarchical Regional Transformer-based Multiple Instance Learning

arXiv: https://arxiv.org/abs/2308.12634 · HTML: https://mlsanity.com/paper/2308.12634/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [RegionViT: Regional-to-Local Attention for Vision Transformers](https://mlsanity.com/paper/2106.02689/) (2021)
- [Multiple Instance Learning for Digital Pathology: A Review on the State-of-the-Art, Limitations & Future Potential](https://mlsanity.com/paper/2206.04425/) (2022)
- [Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images](https://mlsanity.com/paper/2204.12454/) (2022)
- [A Multi-resolution Model for Histopathology Image Classification and Localization with Multiple Instance Learning](https://mlsanity.com/paper/2011.02679/) (2020)
- [Accounting for Dependencies in Deep Learning Based Multiple Instance Learning for Whole Slide Imaging](https://mlsanity.com/paper/2111.01556/) (2021)
- [Contrastive Transformer-based Multiple Instance Learning for Weakly Supervised Polyp Frame Detection](https://mlsanity.com/paper/2203.12121/) (2022)
- [Patch Transformer for Multi-tagging Whole Slide Histopathology Images](https://mlsanity.com/paper/1906.04151/) (2019)
- [Nested Multiple Instance Learning with Attention Mechanisms](https://mlsanity.com/paper/2111.00947/) (2021)
- [Multi-Attention Multiple Instance Learning](https://mlsanity.com/paper/2112.06071/) (2021)
- [HAMIL: Hierarchical Aggregation-Based Multi-Instance Learning for Microscopy Image Classification](https://mlsanity.com/paper/2103.09764/) (2021)
- [Deep Learning-Based Prediction of Molecular Tumor Biomarkers from H&E: A Practical Review](https://mlsanity.com/paper/2211.14847/) (2022)

## Built on by (most cited first, 4 of 4)

- [Sm: enhanced localization in Multiple Instance Learning for medical imaging classification](https://mlsanity.com/paper/2410.03276/) (2024)
- [CARE: A Molecular-Guided Foundation Model with Adaptive Region Modeling for Whole Slide Image Analysis](https://mlsanity.com/paper/2602.21637/) (2026)
- [Latent Diffusion Models with Image-Derived Annotations for Enhanced AI-Assisted Cancer Diagnosis in Histopathology](https://mlsanity.com/paper/2312.09792/) (2023)
- [Modeling Whole-Slide Images as Dynamic Tumor Microenvironment Fields](https://mlsanity.com/paper/2609.34451/) (2026)

## Abstract (arXiv)

The classification of gigapixel histopathology images with deep multiple instance learning models has become a critical task in digital pathology and precision medicine. In this work, we propose a Transformer-based multiple instance learning approach that replaces the traditional learned attention mechanism with a regional, Vision Transformer inspired self-attention mechanism. We present a method that fuses regional patch information to derive slide-level predictions and show how this regional aggregation can be stacked to hierarchically process features on different distance levels. To increase predictive accuracy, especially for datasets with small, local morphological features, we introduce a method to focus the image processing on high attention regions during inference. Our approach is able to significantly improve performance over the baseline on two histopathology datasets and points towards promising directions for further research.