# Diagnose Like a Pathologist: Transformer-Enabled Hierarchical Attention-Guided Multiple Instance Learning for Whole Slide Image Classification

arXiv: https://arxiv.org/abs/2301.08125 · HTML: https://mlsanity.com/paper/2301.08125/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 55 references that are in the map.
- First posted on arXiv 19 Jan 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [Understanding the Difficulty of Training Transformers](https://mlsanity.com/paper/2004.08249/) (2020)
- [Deep neural network models for computational histopathology: A survey](https://mlsanity.com/paper/1912.12378/) (2019)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [Revisiting Multiple Instance Neural Networks](https://mlsanity.com/paper/1610.02501/) (2016)
- [A graph-transformer for whole slide image classification](https://mlsanity.com/paper/2205.09671/) (2022)
- [Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images](https://mlsanity.com/paper/2001.01599/) (2020)
- [Deep Learning for Whole Slide Image Analysis: An Overview](https://mlsanity.com/paper/1910.11097/) (2019)
- [Smooth Loss Functions for Deep Top-k Classification](https://mlsanity.com/paper/1802.07595/) (2018)
- [A Generalized Deep Learning Framework for Whole-Slide Image Segmentation and Analysis](https://mlsanity.com/paper/2001.00258/) (2020)
- [A Comprehensive Review of Computer-aided Whole-slide Image Analysis: from Datasets to Feature Extraction, Segmentation, Classification, and Detection Approaches](https://mlsanity.com/paper/2102.10553/) (2021)
- [Adaptive Weighting Multi-Field-of-View CNN for Semantic Segmentation in Pathology](https://mlsanity.com/paper/1904.06040/) (2019)
- [Semi-Supervised Histology Classification using Deep Multiple Instance Learning and Contrastive Predictive Coding](https://mlsanity.com/paper/1910.10825/) (2019)
- [A Multi-resolution Model for Histopathology Image Classification and Localization with Multiple Instance Learning](https://mlsanity.com/paper/2011.02679/) (2020)
- [Accounting for Dependencies in Deep Learning Based Multiple Instance Learning for Whole Slide Imaging](https://mlsanity.com/paper/2111.01556/) (2021)
- [A Comprehensive Review for Breast Histopathology Image Analysis Using Classical and Deep Neural Networks](https://mlsanity.com/paper/2003.12255/) (2020)
- [Deep Instance-Level Hard Negative Mining Model for Histopathology Images](https://mlsanity.com/paper/1906.09681/) (2019)

## Built on by (most cited first, 27 of 27)

- [Attention-Challenging Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2311.07125/) (2023)
- [MoME: Mixture of Multimodal Experts for Cancer Survival Prediction](https://mlsanity.com/paper/2406.09696/) (2024)
- [FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification](https://mlsanity.com/paper/2411.14743/) (2024)
- [A Survey of Pathology Foundation Model: Progress and Future Directions](https://mlsanity.com/paper/2504.04045/) (2025)
- [Rotation-Agnostic Image Representation Learning for Digital Pathology](https://mlsanity.com/paper/2311.08359/) (2023)
- [Dynamic Policy-Driven Adaptive Multi-Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2403.07939/) (2024)
- [Cross-scale Multi-instance Learning for Pathological Image Diagnosis](https://mlsanity.com/paper/2304.00216/) (2023)
- [AEM: Attention Entropy Maximization for Multiple Instance Learning based Whole Slide Image Classification](https://mlsanity.com/paper/2406.15303/) (2024)
- [Sm: enhanced localization in Multiple Instance Learning for medical imaging classification](https://mlsanity.com/paper/2410.03276/) (2024)
- [Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Bag-Level Classifier is a Good Instance-Level Teacher](https://mlsanity.com/paper/2312.01099/) (2023)
- [TAKT: Target-Aware Knowledge Transfer for Whole Slide Image Classification](https://mlsanity.com/paper/2303.05780/) (2023)
- [RetMIL: Retentive Multiple Instance Learning for Histopathological Whole Slide Image Classification](https://mlsanity.com/paper/2403.10858/) (2024)
- [Integrative Graph-Transformer Framework for Histopathology Whole Slide Image Representation and Classification](https://mlsanity.com/paper/2403.18134/) (2024)
- [CLASS-M: Adaptive stain separation-based contrastive learning with pseudo-labeling for histopathological image classification](https://mlsanity.com/paper/2312.06978/) (2023)
- [RoFormer for Position Aware Multiple Instance Learning in Whole Slide Image Classification](https://mlsanity.com/paper/2310.01924/) (2023)
- [Diagnostic Text-guided Representation Learning in Hierarchical Classification for Pathological Whole Slide Image](https://mlsanity.com/paper/2411.10709/) (2024)
- [Cracking Instance Jigsaw Puzzles: An Alternative to Multiple Instance Learning for Whole Slide Image Analysis](https://mlsanity.com/paper/2507.08178/) (2025)
- [Thinking in Scales: Accelerating Gigapixel Pathology Image Analysis via Adaptive Continuous Reasoning](https://mlsanity.com/paper/2605.19491/) (2026)
- [Deep Multiple Instance Learning with Distance-Aware Self-Attention](https://mlsanity.com/paper/2305.10552/) (2023)
- [Mixing Histopathology Prototypes into Robust Slide-Level Representations for Cancer Subtyping](https://mlsanity.com/paper/2310.12769/) (2023)
- [From Pixels to Gigapixels: Bridging Local Inductive Bias and Long-Range Dependencies with Pixel-Mamba](https://mlsanity.com/paper/2412.16711/) (2024)
- [Exploiting Low-Dimensional Manifold of Features for Few-Shot Whole Slide Image Classification](https://mlsanity.com/paper/2505.15504/) (2025)
- [Towards Efficient Information Fusion: Concentric Dual Fusion Attention Based Multiple Instance Learning for Whole Slide Images](https://mlsanity.com/paper/2403.14346/) (2024)
- [Diagnose Like A REAL Pathologist: An Uncertainty-Focused Approach for Trustworthy Multi-Resolution Multiple Instance Learning](https://mlsanity.com/paper/2511.06433/) (2025)
- [Learning from the Right Patches: A Two-Stage Wavelet-Driven Masked Autoencoder for Histopathology Representation Learning](https://mlsanity.com/paper/2511.06958/) (2025)
- [Enabling Progressive Whole-slide Image Analysis with Multi-scale Pyramidal Network](https://mlsanity.com/paper/2602.01951/) (2026)
- [A Multi-scale Linear-time Encoder for Whole-Slide Image Analysis](https://mlsanity.com/paper/2602.02918/) (2026)

## Abstract (arXiv)

Multiple Instance Learning (MIL) and transformers are increasingly popular in histopathology Whole Slide Image (WSI) classification. However, unlike human pathologists who selectively observe specific regions of histopathology tissues under different magnifications, most methods do not incorporate multiple resolutions of the WSIs, hierarchically and attentively, thereby leading to a loss of focus on the WSIs and information from other resolutions. To resolve this issue, we propose a Hierarchical Attention-Guided Multiple Instance Learning framework to fully exploit the WSIs. This framework can dynamically and attentively discover the discriminative regions across multiple resolutions of the WSIs. Within this framework, an Integrated Attention Transformer is proposed to further enhance the performance of the transformer and obtain a more holistic WSI (bag) representation. This transformer consists of multiple Integrated Attention Modules, which is the combination of a transformer layer and an aggregation module that produces a bag representation based on every instance representation in that bag. The experimental results show that our method achieved state-of-the-art performances on multiple datasets, including Camelyon16, TCGA-RCC, TCGA-NSCLC, and an in-house IMGC dataset. The code is available at https://github.com/BearCleverProud/HAG-MIL.