# Deep Visual Attention Prediction

arXiv: https://arxiv.org/abs/1705.02544 · HTML: https://mlsanity.com/paper/1705.02544/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2017.

## Built on by (most cited first, 30 of 70)

- [Gaze Estimation using Transformer](https://mlsanity.com/paper/2105.14424/) (2021)
- [PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation](https://mlsanity.com/paper/2103.13173/) (2021)
- [TranSalNet: Towards perceptually relevant visual saliency prediction](https://mlsanity.com/paper/2110.03593/) (2021)
- [MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1912.07773/) (2019)
- [ScanGAN360: A Generative Model of Realistic Scanpaths for 360circ Images](https://mlsanity.com/paper/2103.13922/) (2021)
- [Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers](https://mlsanity.com/paper/2303.09383/) (2023)
- [Spatio-Temporal Self-Attention Network for Video Saliency Prediction](https://mlsanity.com/paper/2108.10696/) (2021)
- [CASP-Net: Rethinking Video Saliency Prediction from an Audio-VisualConsistency Perceptual Perspective](https://mlsanity.com/paper/2303.06357/) (2023)
- [DVGaze: Dual-View Gaze Estimation](https://mlsanity.com/paper/2308.10310/) (2023)
- [Personal Fixations-Based Object Segmentation with Object Localization and Boundary Preservation](https://mlsanity.com/paper/2101.09014/) (2021)
- [PSNet: Parallel Symmetric Network for Video Salient Object Detection](https://mlsanity.com/paper/2210.05912/) (2022)
- [SUM: Saliency Unification through Mamba for Visual Attention Modeling](https://mlsanity.com/paper/2406.17815/) (2024)
- [MTGLS: Multi-Task Gaze Estimation with Limited Supervision](https://mlsanity.com/paper/2110.12100/) (2021)
- [Instance-Level Relative Saliency Ranking with Graph Reasoning](https://mlsanity.com/paper/2107.03824/) (2021)
- [SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks](https://mlsanity.com/paper/2112.03731/) (2021)
- [TinyHD: Efficient Video Saliency Prediction with Heterogeneous Decoders using Hierarchical Maps Distillation](https://mlsanity.com/paper/2301.04619/) (2023)
- [Attention mechanisms and deep learning for machine vision: A survey of the state of the art](https://mlsanity.com/paper/2106.07550/) (2021)
- [Exploring Diverse Representations for Open Set Recognition](https://mlsanity.com/paper/2401.06521/) (2024)
- [(ASNA) An Attention-based Siamese-Difference Neural Network with Surrogate Ranking Loss function for Perceptual Image Quality Assessment](https://mlsanity.com/paper/2105.02531/) (2021)
- [Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents](https://mlsanity.com/paper/2201.02569/) (2022)
- [UniAR: A Unified model for predicting human Attention and Responses on visual content](https://mlsanity.com/paper/2312.10175/) (2023)
- [Learning to Predict Salient Faces: A Novel Visual-Audio Saliency Model](https://mlsanity.com/paper/2103.15438/) (2021)
- [Efficient Anomaly Detection Using Self-Supervised Multi-Cue Tasks](https://mlsanity.com/paper/2111.12379/) (2021)
- [Human Attention-Guided Explainable Artificial Intelligence for Computer Vision Models](https://mlsanity.com/paper/2305.03601/) (2023)
- [Depth as Attention for Face Representation Learning](https://mlsanity.com/paper/2101.00652/) (2021)
- [Rethinking Object Saliency Ranking: A Novel Whole-flow Processing Paradigm](https://mlsanity.com/paper/2312.03226/) (2023)
- [How Does Audio Influence Visual Attention in Omnidirectional Videos? Database and Model](https://mlsanity.com/paper/2408.05411/) (2024)
- [A Gated Fusion Network for Dynamic Saliency Prediction](https://mlsanity.com/paper/2102.07682/) (2021)
- [Bio-Inspired Representation Learning for Visual Attention Prediction](https://mlsanity.com/paper/2103.05310/) (2021)
- [Predicting Visual Attention in Graphic Design Documents](https://mlsanity.com/paper/2407.02439/) (2024)

## Abstract (arXiv)

In this work, we aim to predict human eye fixation with view-free scenes based on an end-to-end deep learning architecture. Although Convolutional Neural Networks (CNNs) have made substantial improvement on human attention prediction, it is still needed to improve CNN based attention models by efficiently leveraging multi-scale features. Our visual attention network is proposed to capture hierarchical saliency information from deep, coarse layers with global saliency information to shallow, fine layers with local saliency response. Our model is based on a skip-layer network structure, which predicts human attention from multiple convolutional layers with various reception fields. Final saliency prediction is achieved via the cooperation of those global and local predictions. Our model is learned in a deep supervision manner, where supervision is directly fed into multi-level layers, instead of previous approaches of providing supervision only at the output layer and propagating this supervision back to earlier layers. Our model thus incorporates multi-level saliency predictions within a single network, which significantly decreases the redundancy of previous approaches of learning multiple network streams with different input scales. Extensive experimental analysis on various challenging benchmark datasets demonstrate our method yields state-of-the-art performance with competitive inference time.