# UnOVOST: Unsupervised Offline Video Object Segmentation and Tracking

arXiv: https://arxiv.org/abs/2001.05425 · HTML: https://mlsanity.com/paper/2001.05425/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2020.

## Built on by (most cited first, 30 of 34)

- [Tracking Anything with Decoupled Video Segmentation](https://mlsanity.com/paper/2309.03903/) (2023)
- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [A Generalist Framework for Panoptic Segmentation of Images and Videos](https://mlsanity.com/paper/2210.06366/) (2022)
- [Video Instance Segmentation with a Propose-Reduce Paradigm](https://mlsanity.com/paper/2103.13746/) (2021)
- [BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video](https://mlsanity.com/paper/2209.12118/) (2022)
- [Opening up Open-World Tracking](https://mlsanity.com/paper/2104.11221/) (2021)
- [Self-supervised Object-Centric Learning for Videos](https://mlsanity.com/paper/2310.06907/) (2023)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation](https://mlsanity.com/paper/2308.14710/) (2023)
- [Moving Object Segmentation: All You Need Is SAM (and Flow)](https://mlsanity.com/paper/2404.12389/) (2024)
- [Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentation](https://mlsanity.com/paper/2104.04782/) (2021)
- [Weakly Supervised Instance Segmentation for Videos with Temporal Mask Consistency](https://mlsanity.com/paper/2103.12886/) (2021)
- [UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation via Segment Anything Model](https://mlsanity.com/paper/2305.12659/) (2023)
- [Appearance-Based Refinement for Object-Centric Motion Segmentation](https://mlsanity.com/paper/2312.11463/) (2023)
- [DG-Labeler and DGL-MOTS Dataset: Boost the Autonomous Driving Perception](https://mlsanity.com/paper/2110.07790/) (2021)
- [Learning to Better Segment Objects from Unseen Classes with Unlabeled Videos](https://mlsanity.com/paper/2104.12276/) (2021)
- [Co-attention Propagation Network for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2304.03910/) (2023)
- [CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2311.00987/) (2023)
- [Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2309.11707/) (2023)
- [Out of Sight, Still in Mind: Reasoning and Planning about Unobserved Objects with Video Tracking Enabled Memory Models](https://mlsanity.com/paper/2309.15278/) (2023)
- [STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots](https://mlsanity.com/paper/2311.02337/) (2023)
- [Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentation](https://mlsanity.com/paper/2203.01217/) (2022)
- [MOFO: MOtion FOcused Self-Supervision for Video Understanding](https://mlsanity.com/paper/2308.12447/) (2023)
- [Weakly Supervised Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2101.00667/) (2021)
- [Hierarchical Graph Pattern Understanding for Zero-Shot VOS](https://mlsanity.com/paper/2312.09525/) (2023)
- [Maximal Cliques on Multi-Frame Proposal Graph for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2301.12352/) (2023)
- [MED-VT++: Unifying Multimodal Learning with a Multiscale Encoder-Decoder Video Transformer](https://mlsanity.com/paper/2304.05930/) (2023)
- [Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separation](https://mlsanity.com/paper/2312.04167/) (2023)
- [Learning Appearance and Motion Cues for Panoptic Tracking](https://mlsanity.com/paper/2503.09191/) (2025)
- [Unsupervised Multiple-Object Tracking with a Dynamical Variational Autoencoder](https://mlsanity.com/paper/2202.09315/) (2022)

## Abstract (arXiv)

We address Unsupervised Video Object Segmentation (UVOS), the task of automatically generating accurate pixel masks for salient objects in a video sequence and of tracking these objects consistently through time, without any input about which objects should be tracked. Towards solving this task, we present UnOVOST (Unsupervised Offline Video Object Segmentation and Tracking) as a simple and generic algorithm which is able to track and segment a large variety of objects. This algorithm builds up tracks in a number stages, first grouping segments into short tracklets that are spatio-temporally consistent, before merging these tracklets into long-term consistent object tracks based on their visual similarity. In order to achieve this we introduce a novel tracklet-based Forest Path Cutting data association algorithm which builds up a decision forest of track hypotheses before cutting this forest into paths that form long-term consistent object tracks. When evaluating our approach on the DAVIS 2017 Unsupervised dataset we obtain state-of-the-art performance with a mean J &F score of 67.9% on the val, 58% on the test-dev and 56.4% on the test-challenge benchmarks, obtaining first place in the DAVIS 2019 Unsupervised Video Object Segmentation Challenge. UnOVOST even performs competitively with many semi-supervised video object segmentation algorithms even though it is not given any input as to which objects should be tracked and segmented.