# State-of-the-Art in Human Scanpath Prediction

arXiv: https://arxiv.org/abs/2102.12239 · HTML: https://mlsanity.com/paper/2102.12239/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 59 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Builds on (most cited first, 9 of 9)

- [TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking](https://mlsanity.com/paper/1504.06755/) (2015)
- [Learning to Segment Every Thing](https://mlsanity.com/paper/1711.10370/) (2017)
- [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning](https://mlsanity.com/paper/2005.14310/) (2020)
- [CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research](https://mlsanity.com/paper/1505.03581/) (2015)
- [PathGAN: Visual Scanpath Prediction with Generative Adversarial Networks](https://mlsanity.com/paper/1809.00567/) (2018)
- [SaltiNet: Scan-path Prediction on 360 Degree Images using Saliency Volumes](https://mlsanity.com/paper/1707.03123/) (2017)
- [Spatial statistics and attentional dynamics in scene viewing](https://mlsanity.com/paper/1405.3270/) (2014)
- [Likelihood-based Parameter Estimation and Comparison of Dynamical Cognitive Models](https://mlsanity.com/paper/1606.07309/) (2016)
- [Searchers adjust their eye movement dynamics to the target characteristics in natural scenes](https://mlsanity.com/paper/1802.04069/) (2018)

## Built on by (most cited first, 22 of 22)

- [Automatic Gaze Analysis: A Survey of Deep Learning based Approaches](https://mlsanity.com/paper/2108.05479/) (2021)
- [ScanGAN360: A Generative Model of Realistic Scanpaths for 360circ Images](https://mlsanity.com/paper/2103.13922/) (2021)
- [Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers](https://mlsanity.com/paper/2303.09383/) (2023)
- [Target-absent Human Attention](https://mlsanity.com/paper/2207.01166/) (2022)
- [Eyettention: An Attention-based Dual-Sequence Model for Predicting Human Scanpaths during Reading](https://mlsanity.com/paper/2304.10784/) (2023)
- [Synthesizing Human Gaze Feedback for Improved NLP Performance](https://mlsanity.com/paper/2302.05721/) (2023)
- [ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Texts](https://mlsanity.com/paper/2310.15587/) (2023)
- [Behind the Machine's Gaze: Neural Networks with Biologically-inspired Constraints Exhibit Human-like Visual Attention](https://mlsanity.com/paper/2204.09093/) (2022)
- [Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction](https://mlsanity.com/paper/2507.23021/) (2025)
- [DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Images](https://mlsanity.com/paper/2509.16767/) (2025)
- [Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts](https://mlsanity.com/paper/2207.04250/) (2022)
- [TPP-Gaze: Modelling Gaze Dynamics in Space and Time with Neural Temporal Point Processes](https://mlsanity.com/paper/2410.23409/) (2024)
- [A Probabilistic Time-Evolving Approach to Scanpath Prediction](https://mlsanity.com/paper/2204.09404/) (2022)
- [Modeling Human Eye Movements with Neural Networks in a Maze-Solving Task](https://mlsanity.com/paper/2212.10367/) (2022)
- [Impact of Design Decisions in Scanpath Modeling](https://mlsanity.com/paper/2405.08981/) (2024)
- [Human Scanpath Prediction in Target-Present Visual Search with Semantic-Foveal Bayesian Attention](https://mlsanity.com/paper/2507.18503/) (2025)
- [Semantic-Based Active Perception for Humanoid Visual Tasks with Foveal Sensors](https://mlsanity.com/paper/2404.10836/) (2024)
- [Unified Dynamic Scanpath Predictors Outperform Individually Trained Neural Models](https://mlsanity.com/paper/2405.02929/) (2024)
- [Unified Attention Modeling for Efficient Free-Viewing and Visual Search via Shared Representations](https://mlsanity.com/paper/2506.02764/) (2025)
- [DeepGaze3.5-VL: Modeling Scanpaths via Autoregressive Token Prediction](https://mlsanity.com/paper/2607.02083/) (2026)
- [Visual Attention Graph](https://mlsanity.com/paper/2503.08531/) (2025)
- [Saccade crossing avoidance as a visual search strategy](https://mlsanity.com/paper/2508.18404/) (2025)

## Abstract (arXiv)

The last years have seen a surge in models predicting the scanpaths of fixations made by humans when viewing images. However, the field is lacking a principled comparison of those models with respect to their predictive power. In the past, models have usually been evaluated based on comparing human scanpaths to scanpaths generated from the model. Here, instead we evaluate models based on how well they predict each fixation in a scanpath given the previous scanpath history. This makes model evaluation closely aligned with the biological processes thought to underly scanpath generation and allows to apply established saliency metrics like AUC and NSS in an intuitive and interpretable way. We evaluate many existing models of scanpath prediction on the datasets MIT1003, MIT300, CAT2000 train and CAT200 test, for the first time giving a detailed picture of the current state of the art of human scanpath prediction. We also show that the discussed method of model benchmarking allows for more detailed analyses leading to interesting insights about where and when models fail to predict human behaviour. The MIT/Tuebingen Saliency Benchmark will implement the evaluation of scanpath models as detailed here, allowing researchers to score their models on the established benchmark datasets MIT300 and CAT2000.