# Shallow and Deep Convolutional Networks for Saliency Prediction

arXiv: https://arxiv.org/abs/1603.00845 · HTML: https://mlsanity.com/paper/1603.00845/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2016.

## Built on by (most cited first, 30 of 42)

- [Goal-Oriented Gaze Estimation for Zero-Shot Learning](https://mlsanity.com/paper/2103.03433/) (2021)
- [End-to-End Human-Gaze-Target Detection with Transformers](https://mlsanity.com/paper/2203.10433/) (2022)
- [Voila-A: Aligning Vision-Language Models with User's Gaze Attention](https://mlsanity.com/paper/2401.09454/) (2023)
- [ScanGAN360: A Generative Model of Realistic Scanpaths for 360circ Images](https://mlsanity.com/paper/2103.13922/) (2021)
- [Spatio-Temporal Self-Attention Network for Video Saliency Prediction](https://mlsanity.com/paper/2108.10696/) (2021)
- ["BNN - BN = ?": Training Binary Neural Networks without Batch Normalization](https://mlsanity.com/paper/2104.08215/) (2021)
- [MAAD: A Model and Dataset for "Attended Awareness" in Driving](https://mlsanity.com/paper/2110.08610/) (2021)
- [SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks](https://mlsanity.com/paper/2112.03731/) (2021)
- [Attention mechanisms and deep learning for machine vision: A survey of the state of the art](https://mlsanity.com/paper/2106.07550/) (2021)
- [SALYPATH: A Deep-Based Architecture for visual attention prediction](https://mlsanity.com/paper/2107.00559/) (2021)
- [Joint Gaze-Location and Gaze-Object Detection](https://mlsanity.com/paper/2308.13857/) (2023)
- [Physics-informed ConvNet: Learning Physical Field from a Shallow Neural Network](https://mlsanity.com/paper/2201.10967/) (2022)
- [Data Augmentation via Latent Diffusion for Saliency Prediction](https://mlsanity.com/paper/2409.07307/) (2024)
- [Correcting Face Distortion in Wide-Angle Videos](https://mlsanity.com/paper/2111.09950/) (2021)
- [Multi-task UNet: Jointly Boosting Saliency Prediction and Disease Classification on Chest X-ray Images](https://mlsanity.com/paper/2202.07118/) (2022)
- [Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts](https://mlsanity.com/paper/2207.04250/) (2022)
- [Scanpath Prediction in Panoramic Videos via Expected Code Length Minimization](https://mlsanity.com/paper/2305.02536/) (2023)
- [Learning User Embeddings from Human Gaze for Personalised Saliency Prediction](https://mlsanity.com/paper/2403.13653/) (2024)
- [How Does Audio Influence Visual Attention in Omnidirectional Videos? Database and Model](https://mlsanity.com/paper/2408.05411/) (2024)
- [A Gated Fusion Network for Dynamic Saliency Prediction](https://mlsanity.com/paper/2102.07682/) (2021)
- [Bio-Inspired Representation Learning for Visual Attention Prediction](https://mlsanity.com/paper/2103.05310/) (2021)
- [Where to look at the movies : Analyzing visual attention to understand movie editing](https://mlsanity.com/paper/2102.13378/) (2021)
- [Modeling Object Dissimilarity for Deep Saliency Prediction](https://mlsanity.com/paper/2104.03864/) (2021)
- [Audio-Visual Collaborative Representation Learning for Dynamic Saliency Prediction](https://mlsanity.com/paper/2109.08371/) (2021)
- [A Probabilistic Time-Evolving Approach to Scanpath Prediction](https://mlsanity.com/paper/2204.09404/) (2022)
- [Learning Saliency From Fixations](https://mlsanity.com/paper/2311.14073/) (2023)
- [A Simple and efficient deep Scanpath Prediction](https://mlsanity.com/paper/2112.04610/) (2021)
- [Weakly Supervised Visual-Auditory Fixation Prediction with Multigranularity Perception](https://mlsanity.com/paper/2112.13697/) (2021)
- [Clustered Saliency Prediction](https://mlsanity.com/paper/2207.02205/) (2022)
- [Multi-Scale Estimation for Omni-Directional Saliency Maps Using Learnable Equator Bias](https://mlsanity.com/paper/2309.08139/) (2023)

## Abstract (arXiv)

The prediction of salient areas in images has been traditionally addressed with hand-crafted features based on neuroscience principles. This paper, however, addresses the problem with a completely data-driven approach by training a convolutional neural network (convnet). The learning process is formulated as a minimization of a loss function that measures the Euclidean distance of the predicted saliency map with the provided ground truth. The recent publication of large datasets of saliency prediction has provided enough data to train end-to-end architectures that are both fast and accurate. Two designs are proposed: a shallow convnet trained from scratch, and a another deeper solution whose first three layers are adapted from another network trained for classification. To the authors knowledge, these are the first end-to-end CNNs trained and tested for the purpose of saliency prediction.