# A Benchmark Dataset and Saliency-guided Stacked Autoencoders for Video-based Salient Object Detection

arXiv: https://arxiv.org/abs/1611.00135 · HTML: https://mlsanity.com/paper/1611.00135/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2016.

## Built on by (most cited first, 19 of 19)

- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [Weakly Supervised Video Salient Object Detection](https://mlsanity.com/paper/2104.02391/) (2021)
- [Weakly Supervised Video Salient Object Detection via Point Supervision](https://mlsanity.com/paper/2207.07269/) (2022)
- [PSNet: Parallel Symmetric Network for Video Salient Object Detection](https://mlsanity.com/paper/2210.05912/) (2022)
- [Rapid Salient Object Detection with Difference Convolutional Neural Networks](https://mlsanity.com/paper/2507.01182/) (2025)
- [Depth-Cooperated Trimodal Network for Video Salient Object Detection](https://mlsanity.com/paper/2202.06060/) (2022)
- [Scribble-Supervised RGB-T Salient Object Detection](https://mlsanity.com/paper/2303.09733/) (2023)
- [Salient Object Detection in RGB-D Videos](https://mlsanity.com/paper/2310.15482/) (2023)
- [ViDSOD-100: A New Dataset and a Baseline Model for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2406.12536/) (2024)
- [SHD360: A Benchmark Dataset for Salient Human Detection in 360deg Videos](https://mlsanity.com/paper/2105.11578/) (2021)
- [A Novel Long-term Iterative Mining Scheme for Video Salient Object Detection](https://mlsanity.com/paper/2206.09564/) (2022)
- [ASOD60K: An Audio-Induced Salient Object Detection Dataset for Panoramic Videos](https://mlsanity.com/paper/2107.11629/) (2021)
- [From Sight to Insight: Unleashing Eye-Tracking in Weakly Supervised Video Salient Object Detection](https://mlsanity.com/paper/2506.23519/) (2025)
- [Exploring Driving-aware Salient Object Detection via Knowledge Transfer](https://mlsanity.com/paper/2105.08286/) (2021)
- [TIVE: A Toolbox for Identifying Video Instance Segmentation Errors](https://mlsanity.com/paper/2210.08856/) (2022)
- [Learning Video Salient Object Detection Progressively from Unlabeled Videos](https://mlsanity.com/paper/2204.02008/) (2022)
- [Unleashing the Power of Motion and Depth: A Selective Fusion Strategy for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2507.21857/) (2025)
- [Liquid Fusion of Heterogeneous Representations Towards General Salient Object Detection](https://mlsanity.com/paper/2606.26849/) (2026)
- [RDVSv2: A Large-scale Benchmark for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2607.25392/) (2026)

## Abstract (arXiv)

Image-based salient object detection (SOD) has been extensively studied in the past decades. However, video-based SOD is much less explored since there lack large-scale video datasets within which salient objects are unambiguously defined and annotated. Toward this end, this paper proposes a video-based SOD dataset that consists of 200 videos (64 minutes). In constructing the dataset, we manually annotate all objects and regions over 7,650 uniformly sampled keyframes and collect the eye-tracking data of 23 subjects that free-view all videos. From the user data, we find salient objects in video can be defined as objects that consistently pop-out throughout the video, and objects with such attributes can be unambiguously annotated by combining manually annotated object/region masks with eye-tracking data of multiple subjects. To the best of our knowledge, it is currently the largest dataset for video-based salient object detection. Based on this dataset, this paper proposes an unsupervised baseline approach for video-based SOD by using saliency-guided stacked autoencoders. In the proposed approach, multiple spatiotemporal saliency cues are first extracted at pixel, superpixel and object levels. With these saliency cues, stacked autoencoders are unsupervisedly constructed which automatically infer a saliency score for each pixel by progressively encoding the high-dimensional saliency cues gathered from the pixel and its spatiotemporal neighbors. Experimental results show that the proposed unsupervised approach outperforms 30 state-of-the-art models on the proposed dataset, including 19 image-based & classic (unsupervised or non-deep learning), 6 image-based & deep learning, and 5 video-based & unsupervised. Moreover, benchmarking results show that the proposed dataset is very challenging and has the potential to boost the development of video-based SOD.