# Video Object Segmentation and Tracking: A Survey

arXiv: https://arxiv.org/abs/1904.09172 · HTML: https://mlsanity.com/paper/1904.09172/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2019.

## Built on by (most cited first, 30 of 50)

- [Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing](https://mlsanity.com/paper/2404.14618/) (2024)
- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos](https://mlsanity.com/paper/2409.19603/) (2024)
- [SwiftNet: Real-time Video Object Segmentation](https://mlsanity.com/paper/2102.04604/) (2021)
- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [Learning Object State Changes in Videos: An Open-World Perspective](https://mlsanity.com/paper/2312.11782/) (2023)
- [Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation](https://mlsanity.com/paper/2207.02201/) (2022)
- [Efficient Image Pre-Training with Siamese Cropped Masked Autoencoders](https://mlsanity.com/paper/2403.17823/) (2024)
- [Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview](https://mlsanity.com/paper/2211.08352/) (2022)
- [Perceptual Consistency in Video Segmentation](https://mlsanity.com/paper/2110.12385/) (2021)
- [YOLO-ReT: Towards High Accuracy Real-time Object Detection on Edge GPUs](https://mlsanity.com/paper/2110.13713/) (2021)
- [Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2211.10412/) (2022)
- [Fine-grained Spatiotemporal Grounding on Egocentric Videos](https://mlsanity.com/paper/2508.00518/) (2025)
- [Leveraging VLM-Based Pipelines to Annotate 3D Objects](https://mlsanity.com/paper/2311.17851/) (2023)
- [VideoSeg-R1:Reasoning Video Object Segmentation via Reinforcement Learning](https://mlsanity.com/paper/2511.16077/) (2025)
- [Single Object Tracking: A Survey of Methods, Datasets, and Evaluation Metrics](https://mlsanity.com/paper/2201.13066/) (2022)
- [EVE: Efficient zero-shot text-based Video Editing with Depth Map Guidance and Temporal Consistency Constraints](https://mlsanity.com/paper/2308.10648/) (2023)
- [OSCAR: Object Status and Contextual Awareness for Recipes to Support Non-Visual Cooking](https://mlsanity.com/paper/2503.05962/) (2025)
- [Accelerating Volumetric Medical Image Annotation via Short-Long Memory SAM 2](https://mlsanity.com/paper/2505.01854/) (2025)
- [Fast Interactive Video Object Segmentation with Graph Neural Networks](https://mlsanity.com/paper/2103.03821/) (2021)
- [TDIOT: Target-driven Inference for Deep Video Object Tracking](https://mlsanity.com/paper/2103.11017/) (2021)
- [Optical flow-based branch segmentation for complex orchard environments](https://mlsanity.com/paper/2202.13050/) (2022)
- [CLVOS23: A Long Video Object Segmentation Dataset for Continual Learning](https://mlsanity.com/paper/2304.04259/) (2023)
- [Deep Learning Techniques for Video Instance Segmentation: A Survey](https://mlsanity.com/paper/2310.12393/) (2023)
- [Propagating Semantic Labels in Video Data](https://mlsanity.com/paper/2310.00783/) (2023)
- [MOD-UV: Learning Mobile Object Detectors from Unlabeled Videos](https://mlsanity.com/paper/2405.14841/) (2024)
- [Depthwise-Dilated Convolutional Adapters for Medical Object Tracking and Segmentation Using the Segment Anything Model 2](https://mlsanity.com/paper/2507.14613/) (2025)
- [The SAM2-to-SAM3 Gap in the Segment Anything Model Family: Why Prompt-Based Expertise Fails in Concept-Driven Image Segmentation](https://mlsanity.com/paper/2512.06032/) (2025)
- [Hierarchical Spatiotemporal Transformers for Video Object Segmentation](https://mlsanity.com/paper/2307.08263/) (2023)
- [MSight: An Edge-Cloud Infrastructure-based Perception System for Connected Automated Vehicles](https://mlsanity.com/paper/2310.05290/) (2023)

## Abstract (arXiv)

Object segmentation and object tracking are fundamental research area in the computer vision community. These two topics are diffcult to handle some common challenges, such as occlusion, deformation, motion blur, and scale variation. The former contains heterogeneous object, interacting object, edge ambiguity, and shape complexity. And the latter suffers from difficulties in handling fast motion, out-of-view, and real-time processing. Combining the two problems of video object segmentation and tracking (VOST) can overcome their respective difficulties and improve their performance. VOST can be widely applied to many practical applications such as video summarization, high definition video compression, human computer interaction, and autonomous vehicles. This article aims to provide a comprehensive review of the state-of-the-art tracking methods, and classify these methods into different categories, and identify new trends. First, we provide a hierarchical categorization existing approaches, including unsupervised VOS, semi-supervised VOS, interactive VOS, weakly supervised VOS, and segmentation-based tracking methods. Second, we provide a detailed discussion and overview of the technical characteristics of the different methods. Third, we summarize the characteristics of the related video dataset, and provide a variety of evaluation metrics. Finally, we point out a set of interesting future works and draw our own conclusions.