# Multi-Granularity Video Object Segmentation

arXiv: https://arxiv.org/abs/2412.01471 · HTML: https://mlsanity.com/paper/2412.01471/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 18 of 18)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Personalize Segment Anything Model with One Shot](https://mlsanity.com/paper/2305.03048/) (2023)
- [Associating Objects with Transformers for Video Object Segmentation](https://mlsanity.com/paper/2106.02638/) (2021)
- [ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation](https://mlsanity.com/paper/2012.05258/) (2020)
- [Segment Anything Meets Point Tracking](https://mlsanity.com/paper/2307.01197/) (2023)
- [General Object Foundation Model for Images and Videos at Scale](https://mlsanity.com/paper/2312.09158/) (2023)
- [DVIS++: Improved Decoupled Framework for Universal Video Segmentation](https://mlsanity.com/paper/2312.13305/) (2023)
- [Interactive Video Object Segmentation in the Wild](https://mlsanity.com/paper/1801.00269/) (2017)
- [Vanishing-Point-Guided Video Semantic Segmentation of Driving Scenes](https://mlsanity.com/paper/2401.15261/) (2024)
- [Tracking Anything in High Quality](https://mlsanity.com/paper/2307.13974/) (2023)
- [NOVIS: A Case for End-to-End Near-Online Video Instance Segmentation](https://mlsanity.com/paper/2308.15266/) (2023)
- [Point-VOS: Pointing Up Video Object Segmentation](https://mlsanity.com/paper/2402.05917/) (2024)
- [SAM-PD: How Far Can SAM Take Us in Tracking and Segmenting Anything in Videos by Prompt Denoising](https://mlsanity.com/paper/2403.04194/) (2024)
- [One-Shot Video Inpainting](https://mlsanity.com/paper/2302.14362/) (2023)

## Built on by (most cited first, 1 of 1)

- [SynMulti: Synthetic-to-Real Learning for Multimodal Video Understanding](https://mlsanity.com/paper/2604.12335/) (2026)

## Abstract (arXiv)

Current benchmarks for video segmentation are limited to annotating only salient objects (i.e., foreground instances). Despite their impressive architectural designs, previous works trained on these benchmarks have struggled to adapt to real-world scenarios. Thus, developing a new video segmentation dataset aimed at tracking multi-granularity segmentation target in the video scene is necessary. In this work, we aim to generate multi-granularity video segmentation dataset that is annotated for both salient and non-salient masks. To achieve this, we propose a large-scale, densely annotated multi-granularity video object segmentation (MUG-VOS) dataset that includes various types and granularities of mask annotations. We automatically collected a training set that assists in tracking both salient and non-salient objects, and we also curated a human-annotated test set for reliable evaluation. In addition, we present memory-based mask propagation model (MMPM), trained and evaluated on MUG-VOS dataset, which leads to the best performance among the existing video object segmentation methods and Segment SAM-based video segmentation methods. Project page is available at https://cvlab-kaist.github.io/MUG-VOS.