# Multi-Source Fusion and Automatic Predictor Selection for Zero-Shot Video Object Segmentation

arXiv: https://arxiv.org/abs/2108.05076 · HTML: https://mlsanity.com/paper/2108.05076/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 58 references that are in the map.
- First posted on arXiv 11 Aug 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Multi-scale Interactive Network for Salient Object Detection](https://mlsanity.com/paper/2007.09062/) (2020)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [ParseNet: Looking Wider to See Better](https://mlsanity.com/paper/1506.04579/) (2015)
- [Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation](https://mlsanity.com/paper/1906.03525/) (2019)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Suppress and Balance: A Simple Gated Network for Salient Object Detection](https://mlsanity.com/paper/2007.08074/) (2020)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detection](https://mlsanity.com/paper/1708.02001/) (2017)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks](https://mlsanity.com/paper/2001.06807/) (2020)
- [FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos](https://mlsanity.com/paper/1701.05384/) (2017)
- [Depth-aware CNN for RGB-D Segmentation](https://mlsanity.com/paper/1803.06791/) (2018)
- [Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06227/) (2020)
- [A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection](https://mlsanity.com/paper/2007.06811/) (2020)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation](https://mlsanity.com/paper/1810.01849/) (2018)
- [Learning Motion Patterns in Videos](https://mlsanity.com/paper/1612.07217/) (2016)
- [Learning Discriminative Feature with CRF for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2008.01270/) (2020)
- [CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmark](https://mlsanity.com/paper/1907.00618/) (2019)
- [Video Object Segmentation using Teacher-Student Adaptation in a Human Robot Interaction (HRI) Setting](https://mlsanity.com/paper/1810.07733/) (2018)
- [Is Depth Really Necessary for Salient Object Detection?](https://mlsanity.com/paper/2006.00269/) (2020)
- [EpO-Net: Exploiting Geometric Constraints on Dense Trajectories for Motion Saliency](https://mlsanity.com/paper/1909.13258/) (2019)

## Built on by (most cited first, 18 of 18)

- [Zoom In and Out: A Mixed-scale Triplet Network for Camouflaged Object Detection](https://mlsanity.com/paper/2203.02688/) (2022)
- [CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection](https://mlsanity.com/paper/2112.02363/) (2021)
- [Region Aware Video Object Segmentation with Deep Motion Modeling](https://mlsanity.com/paper/2207.10258/) (2022)
- [Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview](https://mlsanity.com/paper/2211.08352/) (2022)
- [Open-Vocabulary Camouflaged Object Segmentation](https://mlsanity.com/paper/2311.11241/) (2023)
- [Joint Learning of Salient Object Detection, Depth Estimation and Contour Extraction](https://mlsanity.com/paper/2203.04895/) (2022)
- [Promoting Saliency From Depth: Deep Unsupervised RGB-D Saliency Detection](https://mlsanity.com/paper/2205.07179/) (2022)
- [Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation](https://mlsanity.com/paper/2308.06693/) (2023)
- [Depth-Cooperated Trimodal Network for Video Salient Object Detection](https://mlsanity.com/paper/2202.06060/) (2022)
- [A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key!](https://mlsanity.com/paper/2206.13390/) (2022)
- [Adaptive Multi-source Predictor for Zero-shot Video Object Segmentation](https://mlsanity.com/paper/2303.10383/) (2023)
- [CGCOD: Class-Guided Camouflaged Object Detection](https://mlsanity.com/paper/2412.18977/) (2024)
- [Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2501.07806/) (2025)
- [UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation](https://mlsanity.com/paper/2509.16170/) (2025)
- [Hierarchical Graph Pattern Understanding for Zero-Shot VOS](https://mlsanity.com/paper/2312.09525/) (2023)
- [Marine Saliency Segmenter: Object-Focused Conditional Diffusion with Region-Level Semantic Knowledge Distillation](https://mlsanity.com/paper/2504.02391/) (2025)
- [Spatial Semantic Recurrent Mining for Referring Image Segmentation](https://mlsanity.com/paper/2405.09006/) (2024)
- [Towards Imbalanced Motion: Part-Decoupling Network for Video Portrait Segmentation](https://mlsanity.com/paper/2307.16565/) (2023)

## Abstract (arXiv)

Location and appearance are the key cues for video object segmentation. Many sources such as RGB, depth, optical flow and static saliency can provide useful information about the objects. However, existing approaches only utilize the RGB or RGB and optical flow. In this paper, we propose a novel multi-source fusion network for zero-shot video object segmentation. With the help of interoceptive spatial attention module (ISAM), spatial importance of each source is highlighted. Furthermore, we design a feature purification module (FPM) to filter the inter-source incompatible features. By the ISAM and FPM, the multi-source features are effectively fused. In addition, we put forward an automatic predictor selection network (APS) to select the better prediction of either the static saliency predictor or the moving object predictor in order to prevent over-reliance on the failed results caused by low-quality optical flow maps. Extensive experiments on three challenging public benchmarks (i.e. DAVIS16, Youtube-Objects and FBMS) show that the proposed model achieves compelling performance against the state-of-the-arts. The source code will be publicly available at textcolorredurlhttps://github.com/Xiaoqi-Zhao-DLUT/Multi-Source-APS-ZVOS.