# Weakly Supervised Video Salient Object Detection

arXiv: https://arxiv.org/abs/2104.02391 · HTML: https://mlsanity.com/paper/2104.02391/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [EGNet:Edge Guidance Network for Salient Object Detection](https://mlsanity.com/paper/1908.08297/) (2019)
- [One-Shot Video Object Segmentation](https://mlsanity.com/paper/1611.05198/) (2016)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [Visual Saliency Based on Multiscale Deep Features](https://mlsanity.com/paper/1503.08663/) (2015)
- [Learning Features by Watching Objects Move](https://mlsanity.com/paper/1612.06370/) (2016)
- [Context Prior for Scene Segmentation](https://mlsanity.com/paper/2004.01547/) (2020)
- [Normalized Cut Loss for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1804.01346/) (2018)
- [Weakly-Supervised Salient Object Detection via Scribble Annotations](https://mlsanity.com/paper/2003.07685/) (2020)
- [Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks](https://mlsanity.com/paper/2001.06807/) (2020)
- [Motion Guided Attention for Video Salient Object Detection](https://mlsanity.com/paper/1909.07061/) (2019)
- [Motion-Attentive Transition for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2003.04253/) (2020)
- [RANet: Ranking Attention Network for Fast Video Object Segmentation](https://mlsanity.com/paper/1908.06647/) (2019)
- [Making a Case for 3D Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2008.11516/) (2020)
- [Semi-Supervised Video Salient Object Detection Using Pseudo-Labels](https://mlsanity.com/paper/1908.04051/) (2019)
- [Video Salient Object Detection via Fully Convolutional Networks](https://mlsanity.com/paper/1702.00871/) (2017)
- [Learning Video Object Segmentation from Unlabeled Videos](https://mlsanity.com/paper/2003.05020/) (2020)
- [PiCANet: Pixel-wise Contextual Attention Learning for Accurate Saliency Detection](https://mlsanity.com/paper/1812.06314/) (2018)
- [TENet: Triple Excitation Network for Video Salient Object Detection](https://mlsanity.com/paper/2007.09943/) (2020)
- [A Benchmark Dataset and Saliency-guided Stacked Autoencoders for Video-based Salient Object Detection](https://mlsanity.com/paper/1611.00135/) (2016)
- [Unsupervised learning from video to detect foreground objects in single images](https://mlsanity.com/paper/1703.10901/) (2017)
- [A Plug-and-play Scheme to Adapt Image Saliency Deep Model for Video Data](https://mlsanity.com/paper/2008.09103/) (2020)

## Built on by (most cited first, 28 of 28)

- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)
- [Texture-guided Saliency Distilling for Unsupervised Salient Object Detection](https://mlsanity.com/paper/2207.05921/) (2022)
- [A Weakly Supervised Learning Framework for Salient Object Detection via Hybrid Labels](https://mlsanity.com/paper/2209.02957/) (2022)
- [Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation](https://mlsanity.com/paper/2204.02547/) (2022)
- [Weakly Supervised Video Salient Object Detection via Point Supervision](https://mlsanity.com/paper/2207.07269/) (2022)
- [Motion-aware Memory Network for Fast Video Salient Object Detection](https://mlsanity.com/paper/2208.00946/) (2022)
- [PSNet: Parallel Symmetric Network for Video Salient Object Detection](https://mlsanity.com/paper/2210.05912/) (2022)
- [SimulFlow: Simultaneously Extracting Feature and Identifying Target for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2311.18286/) (2023)
- [Boosting Low-Data Instance Segmentation by Unsupervised Pre-training with Saliency Prompt](https://mlsanity.com/paper/2302.01171/) (2023)
- [A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key!](https://mlsanity.com/paper/2206.13390/) (2022)
- [Mutual Information Regularization for Weakly-supervised RGB-D Salient Object Detection](https://mlsanity.com/paper/2306.03630/) (2023)
- [Scribble-Supervised RGB-T Salient Object Detection](https://mlsanity.com/paper/2303.09733/) (2023)
- [Scribble-based Boundary-aware Network for Weakly Supervised Salient Object Detection in Remote Sensing Images](https://mlsanity.com/paper/2202.03501/) (2022)
- [PLGAN: Generative Adversarial Networks for Power-Line Segmentation in Aerial Images](https://mlsanity.com/paper/2204.07243/) (2022)
- [Learning Referring Video Object Segmentation from Weak Annotation](https://mlsanity.com/paper/2308.02162/) (2023)
- [ViDSOD-100: A New Dataset and a Baseline Model for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2406.12536/) (2024)
- [SSFam: Scribble Supervised Salient Object Detection Family](https://mlsanity.com/paper/2409.04817/) (2024)
- [Revisiting Long-tailed Image Classification: Survey and Benchmarks with New Evaluation Metrics](https://mlsanity.com/paper/2302.01507/) (2023)
- [Unified Unsupervised Salient Object Detection via Knowledge Transfer](https://mlsanity.com/paper/2404.14759/) (2024)
- [A Novel Long-term Iterative Mining Scheme for Video Salient Object Detection](https://mlsanity.com/paper/2206.09564/) (2022)
- [CONDA: Condensed Deep Association Learning for Co-Salient Object Detection](https://mlsanity.com/paper/2409.01021/) (2024)
- [From Sight to Insight: Unleashing Eye-Tracking in Weakly Supervised Video Salient Object Detection](https://mlsanity.com/paper/2506.23519/) (2025)
- [SAMSOD: Rethinking SAM Optimization for RGB-T Salient Object Detection](https://mlsanity.com/paper/2510.03689/) (2025)
- [Few-Shot Learning in Video and 3D Object Detection: A Survey](https://mlsanity.com/paper/2507.17079/) (2025)
- [GAPNet: A Lightweight Framework for Image and Video Salient Object Detection via Granularity-Aware Paradigm](https://mlsanity.com/paper/2508.07585/) (2025)
- [Learning Video Salient Object Detection Progressively from Unlabeled Videos](https://mlsanity.com/paper/2204.02008/) (2022)
- [Reliability-Hierarchical Memory Network for Scribble-Supervised Video Object Segmentation](https://mlsanity.com/paper/2303.14384/) (2023)
- [Unleashing the Power of Motion and Depth: A Selective Fusion Strategy for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2507.21857/) (2025)

## Abstract (arXiv)

Significant performance improvement has been achieved for fully-supervised video salient object detection with the pixel-wise labeled training datasets, which are time-consuming and expensive to obtain. To relieve the burden of data annotation, we present the first weakly supervised video salient object detection model based on relabeled "fixation guided scribble annotations". Specifically, an "Appearance-motion fusion module" and bidirectional ConvLSTM based framework are proposed to achieve effective multi-modal learning and long-term temporal context modeling based on our new weak annotations. Further, we design a novel foreground-background similarity loss to further explore the labeling similarity across frames. A weak annotation boosting strategy is also introduced to boost our model performance with a new pseudo-label generation technique. Extensive experimental results on six benchmark video saliency detection datasets illustrate the effectiveness of our solution.