# Modeling Human Visual Search Performance on Realistic Webpages Using Analytical and Deep Learning Methods

arXiv: https://arxiv.org/abs/2005.03784 · HTML: https://mlsanity.com/paper/2005.03784/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2020.

## Built on by (most cited first, 5 of 5)

- [Adapting User Interfaces with Model-based Reinforcement Learning](https://mlsanity.com/paper/2103.06807/) (2021)
- [Predicting Human Performance in Vertical Hierarchical Menu Selection in Immersive AR Using Hand-gesture and Head-gaze](https://mlsanity.com/paper/2206.09480/) (2022)
- [Predicting Visual Attention and Distraction During Visual Search Using Convolutional Neural Networks](https://mlsanity.com/paper/2210.15093/) (2022)
- [AI in HCI Design and User Experience](https://mlsanity.com/paper/2301.00987/) (2023)
- [User-Centered Design (IX): A "User Experience 3.0" Paradigm Framework in the Intelligence Era](https://mlsanity.com/paper/2302.06681/) (2023)

## Abstract (arXiv)

Modeling visual search not only offers an opportunity to predict the usability of an interface before actually testing it on real users, but also advances scientific understanding about human behavior. In this work, we first conduct a set of analyses on a large-scale dataset of visual search tasks on realistic webpages. We then present a deep neural network that learns to predict the scannability of webpage content, i.e., how easy it is for a user to find a specific target. Our model leverages both heuristic-based features such as target size and unstructured features such as raw image pixels. This approach allows us to model complex interactions that might be involved in a realistic visual search task, which can not be easily achieved by traditional analytical models. We analyze the model behavior to offer our insights into how the salience map learned by the model aligns with human intuition and how the learned semantic representation of each target type relates to its visual search performance.