# Iteratively Selecting an Easy Reference Frame Makes Unsupervised Video Object Segmentation Easier

arXiv: https://arxiv.org/abs/2112.12402 · HTML: https://mlsanity.com/paper/2112.12402/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 23 Dec 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Acquisition of Localization Confidence for Accurate Object Detection](https://mlsanity.com/paper/1807.11590/) (2018)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Video Object Segmentation with Episodic Graph Memory Networks](https://mlsanity.com/paper/2007.07020/) (2020)
- [Kernelized Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2007.08270/) (2020)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos](https://mlsanity.com/paper/1701.05384/) (2017)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [Learning Motion Patterns in Videos](https://mlsanity.com/paper/1612.07217/) (2016)
- [Anchor Diffusion for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/1910.10895/) (2019)
- [Learning to Segment Moving Objects](https://mlsanity.com/paper/1712.01127/) (2017)
- [Learning Discriminative Feature with CRF for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2008.01270/) (2020)
- [Progressive Feature Polishing Network for Salient Object Detection](https://mlsanity.com/paper/1911.05942/) (2019)
- [F2Net: Learning to Focus on the Foreground for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2012.02534/) (2020)
- [Video Object Segmentation using Teacher-Student Adaptation in a Human Robot Interaction (HRI) Setting](https://mlsanity.com/paper/1810.07733/) (2018)
- [EpO-Net: Exploiting Geometric Constraints on Dense Trajectories for Motion Saliency](https://mlsanity.com/paper/1909.13258/) (2019)
- [Unsupervised Online Video Object Segmentation with Motion Property Understanding](https://mlsanity.com/paper/1810.03783/) (2018)
- [BubbleNets: Learning to Select the Guidance Frame in Video Object Segmentation by Deep Sorting Frames](https://mlsanity.com/paper/1903.11779/) (2019)

## Built on by (most cited first, 27 of 27)

- [Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2209.03138/) (2022)
- [Unsupervised Video Object Segmentation via Prototype Memory Network](https://mlsanity.com/paper/2209.03712/) (2022)
- [Guided Slot Attention for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2303.08314/) (2023)
- [Dual Prototype Attention for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2211.12036/) (2022)
- [Weakly Supervised Video Salient Object Detection via Point Supervision](https://mlsanity.com/paper/2207.07269/) (2022)
- [SimulFlow: Simultaneously Extracting Feature and Identifying Target for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2311.18286/) (2023)
- [Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation](https://mlsanity.com/paper/2308.06693/) (2023)
- [A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key!](https://mlsanity.com/paper/2206.13390/) (2022)
- [Co-attention Propagation Network for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2304.03910/) (2023)
- [Two-shot Video Object Segmentation](https://mlsanity.com/paper/2303.12078/) (2023)
- [SMITE: Segment Me In TimE](https://mlsanity.com/paper/2410.18538/) (2024)
- [Timeline and Boundary Guided Diffusion Network for Video Shadow Detection](https://mlsanity.com/paper/2408.11785/) (2024)
- [Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2309.11707/) (2023)
- [A Novel Long-term Iterative Mining Scheme for Video Salient Object Detection](https://mlsanity.com/paper/2206.09564/) (2022)
- [Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2501.07806/) (2025)
- [CHROMA: Consistent Harmonization of Multi-View Appearance via Bilateral Grid Prediction](https://mlsanity.com/paper/2507.15748/) (2025)
- [Tsanet: Temporal and Scale Alignment for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2303.04376/) (2023)
- [MED-VT++: Unifying Multimodal Learning with a Multiscale Encoder-Decoder Video Transformer](https://mlsanity.com/paper/2304.05930/) (2023)
- [Improving Unsupervised Video Object Segmentation via Fake Flow Generation](https://mlsanity.com/paper/2407.11714/) (2024)
- [Scoring, Remember, and Reference: Catching Camouflaged Objects in Videos](https://mlsanity.com/paper/2503.17050/) (2025)
- [Towards Imbalanced Motion: Part-Decoupling Network for Video Portrait Segmentation](https://mlsanity.com/paper/2307.16565/) (2023)
- [Treating Motion as Option with Output Selection for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2309.14786/) (2023)
- [Saliency-Motion Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2504.05904/) (2025)
- [MovSAM: A Single-image Moving Object Segmentation Framework Based on Deep Thinking](https://mlsanity.com/paper/2504.06863/) (2025)
- [DepthFlow: Exploiting Depth-Flow Structural Correlations for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2507.19790/) (2025)
- [Shallow Features Matter: Hierarchical Memory with Heterogeneous Interaction for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2507.22465/) (2025)
- [CMTM: Cross-Modal Token Modulation for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2604.14630/) (2026)

## Abstract (arXiv)

Unsupervised video object segmentation (UVOS) is a per-pixel binary labeling problem which aims at separating the foreground object from the background in the video without using the ground truth (GT) mask of the foreground object. Most of the previous UVOS models use the first frame or the entire video as a reference frame to specify the mask of the foreground object. Our question is why the first frame should be selected as a reference frame or why the entire video should be used to specify the mask. We believe that we can select a better reference frame to achieve the better UVOS performance than using only the first frame or the entire video as a reference frame. In our paper, we propose Easy Frame Selector (EFS). The EFS enables us to select an 'easy' reference frame that makes the subsequent VOS become easy, thereby improving the VOS performance. Furthermore, we propose a new framework named as Iterative Mask Prediction (IMP). In the framework, we repeat applying EFS to the given video and selecting an 'easier' reference frame from the video than the previous iteration, increasing the VOS performance incrementally. The IMP consists of EFS, Bi-directional Mask Prediction (BMP), and Temporal Information Updating (TIU). From the proposed framework, we achieve state-of-the-art performance in three UVOS benchmark sets: DAVIS16, FBMS, and SegTrack-V2.