# SimulFlow: Simultaneously Extracting Feature and Identifying Target for Unsupervised Video Object Segmentation

arXiv: https://arxiv.org/abs/2311.18286 · HTML: https://mlsanity.com/paper/2311.18286/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 71 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Builds on (most cited first, 30 of 41)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [One-Shot Video Object Segmentation](https://mlsanity.com/paper/1611.05198/) (2016)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Associating Objects with Transformers for Video Object Segmentation](https://mlsanity.com/paper/2106.02638/) (2021)
- [Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation](https://mlsanity.com/paper/2106.05210/) (2021)
- [FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation](https://mlsanity.com/paper/1902.09513/) (2019)
- [Collaborative Video Object Segmentation by Foreground-Background Integration](https://mlsanity.com/paper/2003.08333/) (2020)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Video Object Segmentation with Episodic Graph Memory Networks](https://mlsanity.com/paper/2007.07020/) (2020)
- [Kernelized Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2007.08270/) (2020)
- [VideoMatch: Matching based Video Object Segmentation](https://mlsanity.com/paper/1809.01123/) (2018)
- [Learning What to Learn for Video Object Segmentation](https://mlsanity.com/paper/2003.11540/) (2020)
- [Efficient Regional Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2103.12934/) (2021)
- [Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks](https://mlsanity.com/paper/2001.06807/) (2020)
- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)
- [Learning Fast and Robust Target Models for Video Object Segmentation](https://mlsanity.com/paper/2003.00908/) (2020)
- [Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning](https://mlsanity.com/paper/1804.03131/) (2018)
- [Hierarchical Memory Matching Network for Video Object Segmentation](https://mlsanity.com/paper/2109.11404/) (2021)
- [A Generative Appearance Model for End-to-end Video Object Segmentation](https://mlsanity.com/paper/1811.11611/) (2018)
- [Making a Case for 3D Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2008.11516/) (2020)
- [Semi-Supervised Video Salient Object Detection Using Pseudo-Labels](https://mlsanity.com/paper/1908.04051/) (2019)
- [Anchor Diffusion for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/1910.10895/) (2019)
- [CNN in MRF: Video Object Segmentation via Inference in A CNN-Based Higher-Order Spatio-Temporal MRF](https://mlsanity.com/paper/1803.09453/) (2018)

## Built on by (most cited first, 12 of 12)

- [OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning](https://mlsanity.com/paper/2403.09634/) (2024)
- [LVOS: A Benchmark for Large-scale Long-term Video Object Segmentation](https://mlsanity.com/paper/2404.19326/) (2024)
- [General Compression Framework for Efficient Transformer Object Tracking](https://mlsanity.com/paper/2409.17564/) (2024)
- [X-Prompt: Multi-modal Visual Prompt for Video Object Segmentation](https://mlsanity.com/paper/2409.19342/) (2024)
- [ClickVOS: Click Video Object Segmentation](https://mlsanity.com/paper/2403.06130/) (2024)
- [MSVCOD:A Large-Scale Multi-Scene Dataset for Video Camouflage Object Detection](https://mlsanity.com/paper/2502.13859/) (2025)
- [Scoring, Remember, and Reference: Catching Camouflaged Objects in Videos](https://mlsanity.com/paper/2503.17050/) (2025)
- [TransFlow: Motion Knowledge Transfer from Video Diffusion Models to Video Salient Object Detection](https://mlsanity.com/paper/2507.19789/) (2025)
- [Saliency-Motion Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2504.05904/) (2025)
- [DepthFlow: Exploiting Depth-Flow Structural Correlations for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2507.19790/) (2025)
- [Shallow Features Matter: Hierarchical Memory with Heterogeneous Interaction for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2507.22465/) (2025)
- [CMTM: Cross-Modal Token Modulation for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2604.14630/) (2026)

## Abstract (arXiv)

Unsupervised video object segmentation (UVOS) aims at detecting the primary objects in a given video sequence without any human interposing. Most existing methods rely on two-stream architectures that separately encode the appearance and motion information before fusing them to identify the target and generate object masks. However, this pipeline is computationally expensive and can lead to suboptimal performance due to the difficulty of fusing the two modalities properly. In this paper, we propose a novel UVOS model called SimulFlow that simultaneously performs feature extraction and target identification, enabling efficient and effective unsupervised video object segmentation. Concretely, we design a novel SimulFlow Attention mechanism to bridege the image and motion by utilizing the flexibility of attention operation, where coarse masks predicted from fused feature at each stage are used to constrain the attention operation within the mask area and exclude the impact of noise. Because of the bidirectional information flow between visual and optical flow features in SimulFlow Attention, no extra hand-designed fusing module is required and we only adopt a light decoder to obtain the final prediction. We evaluate our method on several benchmark datasets and achieve state-of-the-art results. Our proposed approach not only outperforms existing methods but also addresses the computational complexity and fusion difficulties caused by two-stream architectures. Our models achieve 87.4% J & F on DAVIS-16 with the highest speed (63.7 FPS on a 3090) and the lowest parameters (13.7 M). Our SimulFlow also obtains competitive results on video salient object detection datasets.