# Dual Attention Model with Reinforcement Learning for Classification of Histology Whole-Slide Images

arXiv: https://arxiv.org/abs/2302.09682 · HTML: https://mlsanity.com/paper/2302.09682/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 54 references that are in the map.
- First posted on arXiv 19 Feb 2023.

## Builds on (most cited first, 18 of 18)

- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Multiple Instance Learning for Digital Pathology: A Review on the State-of-the-Art, Limitations & Future Potential](https://mlsanity.com/paper/2206.04425/) (2022)
- [Deep Learning for Whole Slide Image Analysis: An Overview](https://mlsanity.com/paper/1910.11097/) (2019)
- [Deep learning-based survival prediction for multiple cancer types using histopathology images](https://mlsanity.com/paper/1912.07354/) (2019)
- [Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images](https://mlsanity.com/paper/2204.12454/) (2022)
- [Deep reinforcement learning in medical imaging: A literature review](https://mlsanity.com/paper/2103.05115/) (2021)
- [Processing Megapixel Images with Deep Attention-Sampling Models](https://mlsanity.com/paper/1905.03711/) (2019)
- [A Comprehensive Review for Breast Histopathology Image Analysis Using Classical and Deep Neural Networks](https://mlsanity.com/paper/2003.12255/) (2020)
- [An Aggregation of Aggregation Methods in Computational Pathology](https://mlsanity.com/paper/2211.01256/) (2022)
- [Her2 Challenge Contest: A Detailed Assessment of Automated Her2 Scoring Algorithms in Whole Slide Images of Breast Cancer Tissues](https://mlsanity.com/paper/1705.08369/) (2017)
- [Reinforcement Learning in Medical Image Analysis: Concepts, Applications, Challenges, and Future Directions](https://mlsanity.com/paper/2206.14302/) (2022)
- [Learning Where to See: A Novel Attention Model for Automated Immunohistochemical Scoring](https://mlsanity.com/paper/1903.10762/) (2019)
- [DSCA: A Dual-Stream Network with Cross-Attention on Whole-Slide Image Pyramids for Cancer Prognosis](https://mlsanity.com/paper/2206.05782/) (2022)
- [Self-Rule to Multi-Adapt: Generalized Multi-source Feature Learning Using Unsupervised Domain Adaptation for Colorectal Cancer Tissue Detection](https://mlsanity.com/paper/2108.09178/) (2021)
- [An End-to-End Breast Tumour Classification Model Using Context-Based Patch Modelling- A BiLSTM Approach for Image Classification](https://mlsanity.com/paper/2106.02864/) (2021)
- [Interpretable HER2 scoring by evaluating clinical Guidelines through a weakly supervised, constrained Deep Learning Approach](https://mlsanity.com/paper/2211.09559/) (2022)

## Built on by (most cited first, 5 of 5)

- [PS3: A Multimodal Transformer Integrating Pathology Reports with Histology Images and Biological Pathways for Cancer Survival Prediction](https://mlsanity.com/paper/2509.20022/) (2025)
- [EvoPS: Evolutionary Patch Selection for Whole Slide Image Analysis in Computational Pathology](https://mlsanity.com/paper/2511.07560/) (2025)
- [DiagR1: A Vision-Language Model Trained via Reinforcement Learning for Digestive Pathology Diagnosis](https://mlsanity.com/paper/2507.18433/) (2025)
- [InfoDPP-PAC: Principled Patch Selection for Whole Slide Image Analysis](https://mlsanity.com/paper/2608.23574/) (2026)
- [Learning Where to Focus: Self-Supervised Multi-Scale ViTs for Histopathology](https://mlsanity.com/paper/2609.18578/) (2026)

## Abstract (arXiv)

Digital whole slide images (WSIs) are generally captured at microscopic resolution and encompass extensive spatial data. Directly feeding these images to deep learning models is computationally intractable due to memory constraints, while downsampling the WSIs risks incurring information loss. Alternatively, splitting the WSIs into smaller patches may result in a loss of important contextual information. In this paper, we propose a novel dual attention approach, consisting of two main components, both inspired by the visual examination process of a pathologist: The first soft attention model processes a low magnification view of the WSI to identify relevant regions of interest, followed by a custom sampling method to extract diverse and spatially distinct image tiles from the selected ROIs. The second component, the hard attention classification model further extracts a sequence of multi-resolution glimpses from each tile for classification. Since hard attention is non-differentiable, we train this component using reinforcement learning to predict the location of the glimpses. This approach allows the model to focus on essential regions instead of processing the entire tile, thereby aligning with a pathologist's way of diagnosis. The two components are trained in an end-to-end fashion using a joint loss function to demonstrate the efficacy of the model. The proposed model was evaluated on two WSI-level classification problems: Human epidermal growth factor receptor 2 scoring on breast cancer histology images and prediction of Intact/Loss status of two Mismatch Repair biomarkers from colorectal cancer histology images. We show that the proposed model achieves performance better than or comparable to the state-of-the-art methods while processing less than 10% of the WSI at the highest magnification and reducing the time required to infer the WSI-level label by more than 75%.