# Adaptive Multi-source Predictor for Zero-shot Video Object Segmentation

arXiv: https://arxiv.org/abs/2303.10383 · HTML: https://mlsanity.com/paper/2303.10383/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 85 references that are in the map.
- First posted on arXiv 18 Mar 2023.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [EGNet:Edge Guidance Network for Salient Object Detection](https://mlsanity.com/paper/1908.08297/) (2019)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [Multi-scale Interactive Network for Salient Object Detection](https://mlsanity.com/paper/2007.09062/) (2020)
- [Deeply supervised salient object detection with short connections](https://mlsanity.com/paper/1611.04849/) (2016)
- [ParseNet: Looking Wider to See Better](https://mlsanity.com/paper/1506.04579/) (2015)
- [F3Net: Fusion, Feedback and Focus for Salient Object Detection](https://mlsanity.com/paper/1911.11445/) (2019)
- [Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation](https://mlsanity.com/paper/1906.03525/) (2019)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks](https://mlsanity.com/paper/1907.06781/) (2019)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [Suppress and Balance: A Simple Gated Network for Salient Object Detection](https://mlsanity.com/paper/2007.08074/) (2020)
- [Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation](https://mlsanity.com/paper/2007.09183/) (2020)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks](https://mlsanity.com/paper/2001.06807/) (2020)
- [JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection](https://mlsanity.com/paper/2004.08515/) (2020)
- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)

## Built on by (most cited first, 7 of 7)

- [Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation](https://mlsanity.com/paper/2308.06693/) (2023)
- [Instance-Level Moving Object Segmentation from a Single Image with Events](https://mlsanity.com/paper/2502.12975/) (2025)
- [UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation](https://mlsanity.com/paper/2509.16170/) (2025)
- [Marine Saliency Segmenter: Object-Focused Conditional Diffusion with Region-Level Semantic Knowledge Distillation](https://mlsanity.com/paper/2504.02391/) (2025)
- [Progressive Confident Masking Attention Network for Audio-Visual Segmentation](https://mlsanity.com/paper/2406.02345/) (2024)
- [Power Battery Detection](https://mlsanity.com/paper/2508.07797/) (2025)
- [DIMOS: Disentangling Instance-level Moving Object Segmentation](https://mlsanity.com/paper/2606.12826/) (2026)

## Abstract (arXiv)

Static and moving objects often occur in real-life videos. Most video object segmentation methods only focus on extracting and exploiting motion cues to perceive moving objects. Once faced with the frames of static objects, the moving object predictors may predict failed results caused by uncertain motion information, such as low-quality optical flow maps. Besides, different sources such as RGB, depth, optical flow and static saliency can provide useful information about the objects. However, existing approaches only consider either the RGB or RGB and optical flow. In this paper, we propose a novel adaptive multi-source predictor for zero-shot video object segmentation (ZVOS). In the static object predictor, the RGB source is converted to depth and static saliency sources, simultaneously. In the moving object predictor, we propose the multi-source fusion structure. First, the spatial importance of each source is highlighted with the help of the interoceptive spatial attention module (ISAM). Second, the motion-enhanced module (MEM) is designed to generate pure foreground motion attention for improving the representation of static and moving features in the decoder. Furthermore, we design a feature purification module (FPM) to filter the inter-source incompatible features. By using the ISAM, MEM and FPM, the multi-source features are effectively fused. In addition, we put forward an adaptive predictor fusion network (APF) to evaluate the quality of the optical flow map and fuse the predictions from the static object predictor and the moving object predictor in order to prevent over-reliance on the failed results caused by low-quality optical flow maps. Experiments show that the proposed model outperforms the state-of-the-art methods on three challenging ZVOS benchmarks. And, the static object predictor precisely predicts a high-quality depth map and static saliency map at the same time.