# Revisiting Video Saliency: A Large-scale Benchmark and a New Model

arXiv: https://arxiv.org/abs/1801.07424 · HTML: https://mlsanity.com/paper/1801.07424/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2018.

## Built on by (most cited first, 30 of 67)

- [Concealed Object Detection](https://mlsanity.com/paper/2102.10274/) (2021)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [Towards Large-Scale Small Object Detection: Survey and Benchmarks](https://mlsanity.com/paper/2207.14096/) (2022)
- [MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1912.07773/) (2019)
- [Temporal-Spatial Feature Pyramid for Video Saliency Detection](https://mlsanity.com/paper/2105.04213/) (2021)
- [Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding](https://mlsanity.com/paper/2303.09706/) (2023)
- [Referring Camouflaged Object Detection](https://mlsanity.com/paper/2306.07532/) (2023)
- [Learning Pixel-Level Distinctions for Video Highlight Detection](https://mlsanity.com/paper/2204.04615/) (2022)
- [CASP-Net: Rethinking Video Saliency Prediction from an Audio-VisualConsistency Perceptual Perspective](https://mlsanity.com/paper/2303.06357/) (2023)
- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [Video Salient Object Detection via Contrastive Features and Attention Modules](https://mlsanity.com/paper/2111.02368/) (2021)
- [DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction](https://mlsanity.com/paper/2403.01226/) (2024)
- [Promoting Saliency From Depth: Deep Unsupervised RGB-D Saliency Detection](https://mlsanity.com/paper/2205.07179/) (2022)
- [ViSaRL: Visual Reinforcement Learning Guided by Human Saliency](https://mlsanity.com/paper/2403.10940/) (2024)
- [CaRDiff: Video Salient Object Ranking Chain of Thought Reasoning for Saliency Prediction with Diffusion](https://mlsanity.com/paper/2408.12009/) (2024)
- [NTIRE 2026 Challenge on Video Saliency Prediction: Methods and Results](https://mlsanity.com/paper/2604.14816/) (2026)
- [A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key!](https://mlsanity.com/paper/2206.13390/) (2022)
- [AIM 2024 Challenge on Video Saliency Prediction: Methods and Results](https://mlsanity.com/paper/2409.14827/) (2024)
- [Learning to Predict Salient Faces: A Novel Visual-Audio Saliency Model](https://mlsanity.com/paper/2103.15438/) (2021)
- [L3GS: Layered 3D Gaussian Splats for Efficient 3D Scene Delivery](https://mlsanity.com/paper/2504.05517/) (2025)
- [A Saliency-Guided Street View Image Inpainting Framework for Efficient Last-Meters Wayfinding](https://mlsanity.com/paper/2205.06934/) (2022)
- [Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts](https://mlsanity.com/paper/2207.04250/) (2022)
- [Towards Stable Co-saliency Detection and Object Co-segmentation](https://mlsanity.com/paper/2209.12138/) (2022)
- [What Do Deep Saliency Models Learn about Visual Attention?](https://mlsanity.com/paper/2310.09679/) (2023)
- [GazeFusion: Saliency-Guided Image Generation](https://mlsanity.com/paper/2407.04191/) (2024)
- [ASOD60K: An Audio-Induced Salient Object Detection Dataset for Panoramic Videos](https://mlsanity.com/paper/2107.11629/) (2021)
- [Predicting Visual Attention in Graphic Design Documents](https://mlsanity.com/paper/2407.02439/) (2024)
- [Text-Audio-Visual-conditioned Diffusion Model for Video Saliency Prediction](https://mlsanity.com/paper/2504.14267/) (2025)
- [Where to look at the movies : Analyzing visual attention to understand movie editing](https://mlsanity.com/paper/2102.13378/) (2021)
- [Region-of-Interest Based Neural Video Compression](https://mlsanity.com/paper/2203.01978/) (2022)

## Abstract (arXiv)

In this work, we contribute to video saliency research in two ways. First, we introduce a new benchmark for predicting human eye movements during dynamic scene free-viewing, which is long-time urged in this field. Our dataset, named DHF1K (Dynamic Human Fixation), consists of 1K high-quality, elaborately selected video sequences spanning a large range of scenes, motions, object types and background complexity. Existing video saliency datasets lack variety and generality of common dynamic scenes and fall short in covering challenging situations in unconstrained environments. In contrast, DHF1K makes a significant leap in terms of scalability, diversity and difficulty, and is expected to boost video saliency modeling. Second, we propose a novel video saliency model that augments the CNN-LSTM network architecture with an attention mechanism to enable fast, end-to-end saliency learning. The attention mechanism explicitly encodes static saliency information, thus allowing LSTM to focus on learning more flexible temporal saliency representation across successive frames. Such a design fully leverages existing large-scale static fixation datasets, avoids overfitting, and significantly improves training efficiency and testing performance. We thoroughly examine the performance of our model, with respect to state-of-the-art saliency models, on three large-scale datasets (i.e., DHF1K, Hollywood2, UCF sports). Experimental results over more than 1.2K testing videos containing 400K frames demonstrate that our model outperforms other competitors.