# Tidying Deep Saliency Prediction Architectures

arXiv: https://arxiv.org/abs/2003.04942 · HTML: https://mlsanity.com/paper/2003.04942/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2020.

## Built on by (most cited first, 20 of 20)

- [TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction](https://mlsanity.com/paper/2301.02315/) (2023)
- [Spatio-Temporal Self-Attention Network for Video Saliency Prediction](https://mlsanity.com/paper/2108.10696/) (2021)
- [Towards Deeper Understanding of Camouflaged Object Detection](https://mlsanity.com/paper/2205.11333/) (2022)
- [Deep Saliency Prior for Reducing Visual Distraction](https://mlsanity.com/paper/2109.01980/) (2021)
- [FBLNet: FeedBack Loop Network for Driver Attention Prediction](https://mlsanity.com/paper/2212.02096/) (2022)
- [SUM: Saliency Unification through Mamba for Visual Attention Modeling](https://mlsanity.com/paper/2406.17815/) (2024)
- [Bringing Generalization to Deep Multi-View Pedestrian Detection](https://mlsanity.com/paper/2109.12227/) (2021)
- [UniAR: A Unified model for predicting human Attention and Responses on visual content](https://mlsanity.com/paper/2312.10175/) (2023)
- [Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction](https://mlsanity.com/paper/2404.14232/) (2024)
- [Data Augmentation via Latent Diffusion for Saliency Prediction](https://mlsanity.com/paper/2409.07307/) (2024)
- [Multi-task UNet: Jointly Boosting Saliency Prediction and Disease Classification on Chest X-ray Images](https://mlsanity.com/paper/2202.07118/) (2022)
- [Blind Surveillance Image Quality Assessment via Deep Neural Network Combined with the Visual Saliency](https://mlsanity.com/paper/2206.04318/) (2022)
- [Enhancing Saliency Prediction in Monitoring Tasks: The Role of Visual Highlights](https://mlsanity.com/paper/2405.09695/) (2024)
- [SalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation](https://mlsanity.com/paper/2407.20062/) (2024)
- [Offline Evaluation of Set-Based Text-to-Image Generation](https://mlsanity.com/paper/2410.17331/) (2024)
- [Unlocking Comics: The AI4VA Dataset for Visual Understanding](https://mlsanity.com/paper/2410.20459/) (2024)
- [Gaze on the Prize: Shaping Visual Attention with Return-Guided Contrastive Learning](https://mlsanity.com/paper/2510.08442/) (2025)
- [Attend to Anything: Foundation Model for Unified Human Attention Modeling](https://mlsanity.com/paper/2606.03540/) (2026)
- [Noise-Aware Video Saliency Prediction](https://mlsanity.com/paper/2104.08038/) (2021)
- [OpenVAM: Open-World Visual Attention Modeling with VLMs](https://mlsanity.com/paper/2609.31364/) (2026)

## Abstract (arXiv)

Learning computational models for visual attention (saliency estimation) is an effort to inch machines/robots closer to human visual cognitive abilities. Data-driven efforts have dominated the landscape since the introduction of deep neural network architectures. In deep learning research, the choices in architecture design are often empirical and frequently lead to more complex models than necessary. The complexity, in turn, hinders the application requirements. In this paper, we identify four key components of saliency models, i.e., input features, multi-level integration, readout architecture, and loss functions. We review the existing state of the art models on these four components and propose novel and simpler alternatives. As a result, we propose two novel end-to-end architectures called SimpleNet and MDNSal, which are neater, minimal, more interpretable and achieve state of the art performance on public saliency benchmarks. SimpleNet is an optimized encoder-decoder architecture and brings notable performance gains on the SALICON dataset (the largest saliency benchmark). MDNSal is a parametric model that directly predicts parameters of a GMM distribution and is aimed to bring more interpretability to the prediction maps. The proposed saliency models can be inferred at 25fps, making them suitable for real-time applications. Code and pre-trained models are available at https://github.com/samyak0210/saliency.