# Semantics-Aware Attention Guidance for Diagnosing Whole Slide Images

arXiv: https://arxiv.org/abs/2404.10894 · HTML: https://mlsanity.com/paper/2404.10894/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [A graph-transformer for whole slide image classification](https://mlsanity.com/paper/2205.09671/) (2022)
- [Accounting for Dependencies in Deep Learning Based Multiple Instance Learning for Whole Slide Imaging](https://mlsanity.com/paper/2111.01556/) (2021)
- [HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis](https://mlsanity.com/paper/2309.07400/) (2023)
- [Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2209.03745/) (2022)

## Built on by (most cited first, 4 of 4)

- [PathFinder: A Multi-Modal Multi-Agent System for Medical Diagnostic Decision-Making Applied to Histopathology](https://mlsanity.com/paper/2502.08916/) (2025)
- [Identifying actionable driver mutations in lung cancer using an efficient Asymmetric Transformer Decoder](https://mlsanity.com/paper/2508.02431/) (2025)
- [WSD-MIL: Window Scale Decay Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2512.19982/) (2025)
- [Normal Guidance is what Attention Needs](https://mlsanity.com/paper/2605.27306/) (2026)

## Abstract (arXiv)

Accurate cancer diagnosis remains a critical challenge in digital pathology, largely due to the gigapixel size and complex spatial relationships present in whole slide images. Traditional multiple instance learning (MIL) methods often struggle with these intricacies, especially in preserving the necessary context for accurate diagnosis. In response, we introduce a novel framework named Semantics-Aware Attention Guidance (SAG), which includes 1) a technique for converting diagnostically relevant entities into attention signals, and 2) a flexible attention loss that efficiently integrates various semantically significant information, such as tissue anatomy and cancerous regions. Our experiments on two distinct cancer datasets demonstrate consistent improvements in accuracy, precision, and recall with two state-of-the-art baseline models. Qualitative analysis further reveals that the incorporation of heuristic guidance enables the model to focus on regions critical for diagnosis. SAG is not only effective for the models discussed here, but its adaptability extends to any attention-based diagnostic model. This opens up exciting possibilities for further improving the accuracy and efficiency of cancer diagnostics.