# DeepGaze II: Reading fixations from deep features trained on object recognition

arXiv: https://arxiv.org/abs/1610.01563 · HTML: https://mlsanity.com/paper/1610.01563/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2016.

## Built on by (most cited first, 30 of 55)

- [Learning Affordance Grounding from Exocentric Images](https://mlsanity.com/paper/2203.09905/) (2022)
- [Affordance Grounding from Demonstration Video to Target Image](https://mlsanity.com/paper/2303.14644/) (2023)
- [Voila-A: Aligning Vision-Language Models with User's Gaze Attention](https://mlsanity.com/paper/2401.09454/) (2023)
- [Learning Visual Affordance Grounding from Demonstration Videos](https://mlsanity.com/paper/2108.05675/) (2021)
- [ScanGAN360: A Generative Model of Realistic Scanpaths for 360circ Images](https://mlsanity.com/paper/2103.13922/) (2021)
- [Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency](https://mlsanity.com/paper/2105.08667/) (2021)
- [Grounded Affordance from Exocentric View](https://mlsanity.com/paper/2208.13196/) (2022)
- [Human Attention in Fine-grained Classification](https://mlsanity.com/paper/2111.01628/) (2021)
- [Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?](https://mlsanity.com/paper/2205.10226/) (2022)
- [GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths](https://mlsanity.com/paper/2408.02788/) (2024)
- [ViSaRL: Visual Reinforcement Learning Guided by Human Saliency](https://mlsanity.com/paper/2403.10940/) (2024)
- [Beyond Average: Individualized Visual Scanpath Prediction](https://mlsanity.com/paper/2404.12235/) (2024)
- [SUM: Saliency Unification through Mamba for Visual Attention Modeling](https://mlsanity.com/paper/2406.17815/) (2024)
- [INTRA: Interaction Relationship-aware Weakly Supervised Affordance Grounding](https://mlsanity.com/paper/2409.06210/) (2024)
- [Better artificial intelligence does not mean better models of biology](https://mlsanity.com/paper/2504.16940/) (2025)
- [Attention mechanisms and deep learning for machine vision: A survey of the state of the art](https://mlsanity.com/paper/2106.07550/) (2021)
- [SALYPATH: A Deep-Based Architecture for visual attention prediction](https://mlsanity.com/paper/2107.00559/) (2021)
- [Behind the Machine's Gaze: Neural Networks with Biologically-inspired Constraints Exhibit Human-like Visual Attention](https://mlsanity.com/paper/2204.09093/) (2022)
- [UniAR: A Unified model for predicting human Attention and Responses on visual content](https://mlsanity.com/paper/2312.10175/) (2023)
- [Brand Visibility in Packaging: A Deep Learning Approach for Logo Detection, Saliency-Map Prediction, and Logo Placement Analysis](https://mlsanity.com/paper/2403.02336/) (2024)
- [Deep Learning--Based Scene Simplification for Bionic Vision](https://mlsanity.com/paper/2102.00297/) (2021)
- [Multi-task UNet: Jointly Boosting Saliency Prediction and Disease Classification on Chest X-ray Images](https://mlsanity.com/paper/2202.07118/) (2022)
- [Understanding and Modeling the Effects of Task and Context on Drivers' Gaze Allocation](https://mlsanity.com/paper/2310.09275/) (2023)
- [What Do Deep Saliency Models Learn about Visual Attention?](https://mlsanity.com/paper/2310.09679/) (2023)
- [Closed-Loop Transfer for Weakly-supervised Affordance Grounding](https://mlsanity.com/paper/2510.17384/) (2025)
- [Predicting Visual Attention in Graphic Design Documents](https://mlsanity.com/paper/2407.02439/) (2024)
- [Modeling Object Dissimilarity for Deep Saliency Prediction](https://mlsanity.com/paper/2104.03864/) (2021)
- [A Probabilistic Time-Evolving Approach to Scanpath Prediction](https://mlsanity.com/paper/2204.09404/) (2022)
- [Context-empowered Visual Attention Prediction in Pedestrian Scenarios](https://mlsanity.com/paper/2210.16933/) (2022)
- [Do MLLMs Exhibit Human-like Perceptual Behaviors? HVSBench: A Benchmark for MLLM Alignment with Human Perceptual Behavior](https://mlsanity.com/paper/2412.09603/) (2024)

## Abstract (arXiv)

Here we present DeepGaze II, a model that predicts where people look in images. The model uses the features from the VGG-19 deep neural network trained to identify objects in images. Contrary to other saliency models that use deep features, here we use the VGG features for saliency prediction with no additional fine-tuning (rather, a few readout layers are trained on top of the VGG features to predict saliency). The model is therefore a strong test of transfer learning. After conservative cross-validation, DeepGaze II explains about 87% of the explainable information gain in the patterns of fixations and achieves top performance in area under the curve metrics on the MIT300 hold-out benchmark. These results corroborate the finding from DeepGaze I (which explained 56% of the explainable information gain), that deep features trained on object recognition provide a versatile feature space for performing related visual tasks. We explore the factors that contribute to this success and present several informative image examples. A web service is available to compute model predictions at http://deepgaze.bethgelab.org.