# Dual Prototype Attention for Unsupervised Video Object Segmentation

arXiv: https://arxiv.org/abs/2211.12036 · HTML: https://mlsanity.com/paper/2211.12036/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 34 references that are in the map.
- First posted on arXiv 22 Nov 2022.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks](https://mlsanity.com/paper/2001.06807/) (2020)
- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)
- [Making a Case for 3D Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2008.11516/) (2020)
- [Anchor Diffusion for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/1910.10895/) (2019)
- [Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2207.08485/) (2022)
- [Learning Discriminative Feature with CRF for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2008.01270/) (2020)
- [Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2209.03138/) (2022)
- [F2Net: Learning to Focus on the Foreground for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2012.02534/) (2020)
- [Unsupervised Video Object Segmentation via Prototype Memory Network](https://mlsanity.com/paper/2209.03712/) (2022)
- [Video Object Segmentation using Teacher-Student Adaptation in a Human Robot Interaction (HRI) Setting](https://mlsanity.com/paper/1810.07733/) (2018)
- [Iteratively Selecting an Easy Reference Frame Makes Unsupervised Video Object Segmentation Easier](https://mlsanity.com/paper/2112.12402/) (2021)
- [D^2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2111.07774/) (2021)

## Built on by (most cited first, 14 of 14)

- [Segment Any Motion in Videos](https://mlsanity.com/paper/2503.22268/) (2025)
- [Appearance-Based Refinement for Object-Centric Motion Segmentation](https://mlsanity.com/paper/2312.11463/) (2023)
- [Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2501.07806/) (2025)
- [Find First, Track Next: Decoupling Identification and Propagation in Referring Video Object Segmentation](https://mlsanity.com/paper/2503.03492/) (2025)
- [VRMDiff: Text-Guided Video Referring Matting Generation of Diffusion](https://mlsanity.com/paper/2503.10678/) (2025)
- [TransFlow: Motion Knowledge Transfer from Video Diffusion Models to Video Salient Object Detection](https://mlsanity.com/paper/2507.19789/) (2025)
- [Bridging Semantic Logic Gaps: A Cognition Inspired Multimodal Boundary Preserving Network for Image Manipulation Localization](https://mlsanity.com/paper/2508.07216/) (2025)
- [GMOS: Grounding Moving Object Segmentation in 3D Space and Time](https://mlsanity.com/paper/2605.30352/) (2026)
- [Treating Motion as Option with Output Selection for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2309.14786/) (2023)
- [Saliency-Motion Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2504.05904/) (2025)
- [SRVP: Strong Recollection Video Prediction Model Using Attention-Based Spatiotemporal Correlation Fusion](https://mlsanity.com/paper/2504.08012/) (2025)
- [DepthFlow: Exploiting Depth-Flow Structural Correlations for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2507.19790/) (2025)
- [FTIO: Frequent Temporally Integrated Objects](https://mlsanity.com/paper/2508.16183/) (2025)
- [M4-SAM: Multi-Modal Mixture-of-Experts with Memory-Augmented SAM for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2605.11760/) (2026)

## Abstract (arXiv)

Unsupervised video object segmentation (VOS) aims to detect and segment the most salient object in videos. The primary techniques used in unsupervised VOS are 1) the collaboration of appearance and motion information; and 2) temporal fusion between different frames. This paper proposes two novel prototype-based attention mechanisms, inter-modality attention (IMA) and inter-frame attention (IFA), to incorporate these techniques via dense propagation across different modalities and frames. IMA densely integrates context information from different modalities based on a mutual refinement. IFA injects global context of a video to the query frame, enabling a full utilization of useful properties from multiple frames. Experimental results on public benchmark datasets demonstrate that our proposed approach outperforms all existing methods by a substantial margin. The proposed two components are also thoroughly validated via ablative study.