# Pay Attention with Focus: A Novel Learning Scheme for Classification of Whole Slide Images

arXiv: https://arxiv.org/abs/2106.06623 · HTML: https://mlsanity.com/paper/2106.06623/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 11 Jun 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [A Survey of the Recent Architectures of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1901.06032/) (2019)
- [Fine-Tuning and Training of DenseNet for Histopathology Image Representation Using TCGA Diagnostic Slides](https://mlsanity.com/paper/2101.07903/) (2021)
- [Deep Learning for Whole Slide Image Analysis: An Overview](https://mlsanity.com/paper/1910.11097/) (2019)
- [Yottixel -- An Image Search Engine for Large Archives of Histopathology Whole Slide Images](https://mlsanity.com/paper/1911.08748/) (2019)
- [Representation Learning of Histopathology Images using Graph Neural Networks](https://mlsanity.com/paper/2004.07399/) (2020)
- [Pan-Cancer Diagnostic Consensus Through Searching Archival Histopathology Images Using Artificial Intelligence](https://mlsanity.com/paper/1911.08736/) (2019)
- [Learning Permutation Invariant Representations using Memory Networks](https://mlsanity.com/paper/1911.07984/) (2019)
- [SD-CNN: a Shallow-Deep CNN for Improved Breast Cancer Diagnosis](https://mlsanity.com/paper/1803.00663/) (2018)
- [Deep Learning with Permutation-invariant Operator for Multi-instance Histopathology Classification](https://mlsanity.com/paper/1712.00310/) (2017)

## Built on by (most cited first, 5 of 5)

- [Handcrafted Histological Transformer (H2T): Unsupervised Representation of Whole Slide Images](https://mlsanity.com/paper/2202.07001/) (2022)
- [ScoreNet: Learning Non-Uniform Attention and Augmentation for Transformer-Based Histopathological Image Classification](https://mlsanity.com/paper/2202.07570/) (2022)
- [Learning Binary and Sparse Permutation-Invariant Representations for Fast and Memory Efficient Whole Slide Image Search](https://mlsanity.com/paper/2208.13653/) (2022)
- [A Short Survey on Set-Based Aggregation Techniques for Single-Vector WSI Representation in Digital Pathology](https://mlsanity.com/paper/2409.04615/) (2024)
- [Aggregation Schemes for Single-Vector WSI Representation Learning in Digital Pathology](https://mlsanity.com/paper/2501.17822/) (2025)

## Abstract (arXiv)

Deep learning methods such as convolutional neural networks (CNNs) are difficult to directly utilize to analyze whole slide images (WSIs) due to the large image dimensions. We overcome this limitation by proposing a novel two-stage approach. First, we extract a set of representative patches (called mosaic) from a WSI. Each patch of a mosaic is encoded to a feature vector using a deep network. The feature extractor model is fine-tuned using hierarchical target labels of WSIs, i.e., anatomic site and primary diagnosis. In the second stage, a set of encoded patch-level features from a WSI is used to compute the primary diagnosis probability through the proposed Pay Attention with Focus scheme, an attention-weighted averaging of predicted probabilities for all patches of a mosaic modulated by a trainable focal factor. Experimental results show that the proposed model can be robust, and effective for the classification of WSIs.