# Measuring and Predicting Where and When Pathologists Focus their Visual Attention while Grading Whole Slide Images of Cancer

arXiv: https://arxiv.org/abs/2508.01668 · HTML: https://mlsanity.com/paper/2508.01668/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 63 references that are in the map.
- First posted on arXiv 3 Aug 2025.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Visual Saliency Based on Multiscale Deep Features](https://mlsanity.com/paper/1503.08663/) (2015)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model](https://mlsanity.com/paper/1611.09571/) (2016)
- [Automated Gleason Grading of Prostate Biopsies using Deep Learning](https://mlsanity.com/paper/1907.07980/) (2019)
- [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning](https://mlsanity.com/paper/2005.14310/) (2020)
- [Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis](https://mlsanity.com/paper/2204.02976/) (2022)
- [Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention](https://mlsanity.com/paper/2303.15274/) (2023)
- [TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction](https://mlsanity.com/paper/2301.02315/) (2023)
- [Top-down Visual Saliency Guided by Captions](https://mlsanity.com/paper/1612.07360/) (2016)
- [Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers](https://mlsanity.com/paper/2303.09383/) (2023)
- [Target-absent Human Attention](https://mlsanity.com/paper/2207.01166/) (2022)
- [Beyond Average: Individualized Visual Scanpath Prediction](https://mlsanity.com/paper/2404.12235/) (2024)
- [Automated Detection of Cribriform Growth Patterns in Prostate Histology Images](https://mlsanity.com/paper/2003.10543/) (2020)
- [Decoding the visual attention of pathologists to reveal their level of expertise](https://mlsanity.com/paper/2403.17255/) (2024)
- [Visual attention analysis of pathologists examining whole slide images of Prostate cancer](https://mlsanity.com/paper/2202.08437/) (2022)
- [A dense subgraph based algorithm for compact salient image region detection](https://mlsanity.com/paper/1511.06545/) (2015)

## Built on by (most cited first, 4 of 4)

- [Personalized Image Descriptions from Attention Sequences](https://mlsanity.com/paper/2512.06662/) (2025)
- [Eye-Tracking, Mouse Tracking, Stimulus Tracking,and Decision-Making Datasets in Digital Pathology](https://mlsanity.com/paper/2510.24653/) (2025)
- [Pathologist Attention-Aligned Report Generation for Prostate Histopathology](https://mlsanity.com/paper/2607.19624/) (2026)
- [PathPilot: An Adaptive Question-Conditioned Multi-Magnification Navigation Agent for Whole-Slide Image Analysis](https://mlsanity.com/paper/2610.16734/) (2026)

## Abstract (arXiv)

The ability to predict the attention of expert pathologists could lead to decision support systems for better pathology training. We developed methods to predict the spatio-temporal (where and when) movements of pathologists' attention as they grade whole slide images (WSIs) of prostate cancer. We characterize a pathologist's attention trajectory by their x, y, and m (magnification) movements of a viewport as they navigate WSIs using a digital microscope. This information was obtained from 43 pathologists across 123 WSIs, and we consider the task of predicting the pathologist attention scanpaths constructed from the viewport centers. We introduce a fixation extraction algorithm that simplifies an attention trajectory by extracting fixations in the pathologist's viewing while preserving semantic information, and we use these pre-processed data to train and test a two-stage model to predict the dynamic (scanpath) allocation of attention during WSI reading via intermediate attention heatmap prediction. In the first stage, a transformer-based sub-network predicts the attention heatmaps (static attention) across different magnifications. In the second stage, we predict the attention scanpath by sequentially modeling the next fixation points in an autoregressive manner using a transformer-based approach, starting at the WSI center and leveraging multi-magnification feature representations from the first stage. Experimental results show that our scanpath prediction model outperforms chance and baseline models. Tools developed from this model could assist pathology trainees in learning to allocate their attention during WSI reading like an expert.