# Benchmarking human visual search computational models in natural scenes: models comparison and reference datasets

arXiv: https://arxiv.org/abs/2112.05808 · HTML: https://mlsanity.com/paper/2112.05808/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 29 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 7 of 7)

- [Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future](https://mlsanity.com/paper/2001.07092/) (2020)
- [Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet](https://mlsanity.com/paper/1411.1045/) (2014)
- [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning](https://mlsanity.com/paper/2005.14310/) (2020)
- [DeepFix: A Fully Convolutional Neural Network for predicting Human Eye Fixations](https://mlsanity.com/paper/1510.02927/) (2015)
- [Finding any Waldo: zero-shot invariant and efficient visual search](https://mlsanity.com/paper/1807.10587/) (2018)
- [Efficient Coarse-to-Fine Non-Local Module for the Detection of Small Objects](https://mlsanity.com/paper/1811.12152/) (2018)
- [Modeling human visual search: A combined Bayesian searcher and saliency map approach for eye movement guidance in natural scenes](https://mlsanity.com/paper/2009.08373/) (2020)

## Abstract (arXiv)

Visual search is an essential part of almost any everyday human goal-directed interaction with the environment. Nowadays, several algorithms are able to predict gaze positions during simple observation, but few models attempt to simulate human behavior during visual search in natural scenes. Furthermore, these models vary widely in their design and exhibit differences in the datasets and metrics with which they were evaluated. Thus, there is a need for a reference point, on which each model can be tested and from where potential improvements can be derived. In the present work, we select publicly available state-of-the-art visual search models in natural scenes and evaluate them on different datasets, employing the same metrics to estimate their efficiency and similarity with human subjects. In particular, we propose an improvement to the Ideal Bayesian Searcher through a combination with a neural network-based visual search model, enabling it to generalize to other datasets. The present work sheds light on the limitations of current models and how potential improvements can be accomplished by combining approaches. Moreover, it moves forward on providing a solution for the urgent need for benchmarking data and metrics to support the development of more general human visual search computational models.