# DVIS++: Improved Decoupled Framework for Universal Video Segmentation

arXiv: https://arxiv.org/abs/2312.13305 · HTML: https://mlsanity.com/paper/2312.13305/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 88 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 15 of 15)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [A Simple Framework for Open-Vocabulary Segmentation and Detection](https://mlsanity.com/paper/2303.08131/) (2023)
- [Semantic-SAM: Segment and Recognize Anything at Any Granularity](https://mlsanity.com/paper/2307.04767/) (2023)
- [Mask2Former for Video Instance Segmentation](https://mlsanity.com/paper/2112.10764/) (2021)
- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentation](https://mlsanity.com/paper/2304.04694/) (2023)
- [GRAtt-VIS: Gated Residual Attention for Auto Rectifying Video Instance Segmentation](https://mlsanity.com/paper/2305.17096/) (2023)
- [NOVIS: A Case for End-to-End Near-Online Video Instance Segmentation](https://mlsanity.com/paper/2308.15266/) (2023)
- [1st Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation](https://mlsanity.com/paper/2306.04091/) (2023)
- [1st Place Solution for the 5th LSVOS Challenge: Video Instance Segmentation](https://mlsanity.com/paper/2308.14392/) (2023)
- [RefineVIS: Video Instance Segmentation with Temporal Attention Refinement](https://mlsanity.com/paper/2306.04774/) (2023)

## Built on by (most cited first, 30 of 46)

- [Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos](https://mlsanity.com/paper/2501.04001/) (2025)
- [VISA: Reasoning Video Object Segmentation via Large Language Models](https://mlsanity.com/paper/2407.11325/) (2024)
- [MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2508.05630/) (2025)
- [DVIS-DAQ: Improving Video Segmentation via Dynamic Anchor Queries](https://mlsanity.com/paper/2404.00086/) (2024)
- [Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding](https://mlsanity.com/paper/2504.10465/) (2025)
- [ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentation](https://mlsanity.com/paper/2412.09754/) (2024)
- [Are They the Same? Exploring Visual Correspondence Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2501.04670/) (2025)
- [SIU3R: Simultaneous Scene Understanding and 3D Reconstruction Beyond Feature Alignment](https://mlsanity.com/paper/2507.02705/) (2025)
- [Zero-Shot Video Semantic Segmentation based on Pre-Trained Diffusion Models](https://mlsanity.com/paper/2405.16947/) (2024)
- [CAVIS: Context-Aware Video Instance Segmentation](https://mlsanity.com/paper/2407.03010/) (2024)
- [Learning Streaming Video Representation via Multitask Training](https://mlsanity.com/paper/2504.20041/) (2025)
- [CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos](https://mlsanity.com/paper/2505.18561/) (2025)
- [Dense360: Dense Understanding from Omnidirectional Panoramas](https://mlsanity.com/paper/2506.14471/) (2025)
- [VidEoMT: Your ViT is Secretly Also a Video Segmentation Model](https://mlsanity.com/paper/2602.17807/) (2026)
- [SMITE: Segment Me In TimE](https://mlsanity.com/paper/2410.18538/) (2024)
- [The 1st Solution for 7th LSVOS RVOS Track: SaSaSa2VA](https://mlsanity.com/paper/2509.16972/) (2025)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [Latest Object Memory Management for Temporally Consistent Video Instance Segmentation](https://mlsanity.com/paper/2507.19754/) (2025)
- [Evaluating SAM2 for Video Semantic Segmentation](https://mlsanity.com/paper/2512.01774/) (2025)
- [Beyond Appearance: Geometric Cues for Robust Video Instance Segmentation](https://mlsanity.com/paper/2507.05948/) (2025)
- [SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding](https://mlsanity.com/paper/2510.13016/) (2025)
- [PMT: Plain Mask Transformer for Image and Video Segmentation with Frozen Vision Encoders](https://mlsanity.com/paper/2603.25398/) (2026)
- [4th PVUW MeViS 3rd Place Report: Sa2VA](https://mlsanity.com/paper/2504.00476/) (2025)
- [FRAME: Pre-Training Video Feature Representations via Anticipation and Memory](https://mlsanity.com/paper/2506.05543/) (2025)
- [A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects](https://mlsanity.com/paper/2506.13552/) (2025)
- [Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2510.10671/) (2025)
- [Multi-Granularity Video Object Segmentation](https://mlsanity.com/paper/2412.01471/) (2024)
- [MoVieDrive: Urban Scene Synthesis with Multi-Modal Multi-View Video Diffusion Transformer](https://mlsanity.com/paper/2508.14327/) (2025)
- [SVAC: Scaling Is All You Need For Referring Video Object Segmentation](https://mlsanity.com/paper/2509.24109/) (2025)
- [ELVIS: Enhance Low-Light for Video Instance Segmentation in the Dark](https://mlsanity.com/paper/2512.01495/) (2025)

## Abstract (arXiv)

We present the Decoupled VIdeo Segmentation (DVIS) framework, a novel approach for the challenging task of universal video segmentation, including video instance segmentation (VIS), video semantic segmentation (VSS), and video panoptic segmentation (VPS). Unlike previous methods that model video segmentation in an end-to-end manner, our approach decouples video segmentation into three cascaded sub-tasks: segmentation, tracking, and refinement. This decoupling design allows for simpler and more effective modeling of the spatio-temporal representations of objects, especially in complex scenes and long videos. Accordingly, we introduce two novel components: the referring tracker and the temporal refiner. These components track objects frame by frame and model spatio-temporal representations based on pre-aligned features. To improve the tracking capability of DVIS, we propose a denoising training strategy and introduce contrastive learning, resulting in a more robust framework named DVIS++. Furthermore, we evaluate DVIS++ in various settings, including open vocabulary and using a frozen pre-trained backbone. By integrating CLIP with DVIS++, we present OV-DVIS++, the first open-vocabulary universal video segmentation framework. We conduct extensive experiments on six mainstream benchmarks, including the VIS, VSS, and VPS datasets. Using a unified architecture, DVIS++ significantly outperforms state-of-the-art specialized methods on these benchmarks in both close- and open-vocabulary settings. Code:~urlhttps://github.com/zhang-tao-whu/DVISPlus.