# 2nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation

arXiv: https://arxiv.org/abs/2406.00500 · HTML: https://mlsanity.com/paper/2406.00500/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 23 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [UPSNet: A Unified Panoptic Segmentation Network](https://mlsanity.com/paper/1901.03784/) (2019)
- [Video Panoptic Segmentation](https://mlsanity.com/paper/2006.11339/) (2020)
- [ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentation](https://mlsanity.com/paper/2012.05258/) (2020)
- [DVIS: Decoupled Video Instance Segmentation Framework](https://mlsanity.com/paper/2306.03413/) (2023)
- [STEP: Segmenting and Tracking Every Pixel](https://mlsanity.com/paper/2102.11859/) (2021)
- [Attention-guided Unified Network for Panoptic Segmentation](https://mlsanity.com/paper/1812.03904/) (2018)
- [An End-to-End Network for Panoptic Segmentation](https://mlsanity.com/paper/1903.05027/) (2019)
- [DVIS++: Improved Decoupled Framework for Universal Video Segmentation](https://mlsanity.com/paper/2312.13305/) (2023)
- [Slot-VPS: Object-centric Representation Learning for Video Panoptic Segmentation](https://mlsanity.com/paper/2112.08949/) (2021)
- [Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentation](https://mlsanity.com/paper/2304.04694/) (2023)
- [ReMaX: Relaxing for Better Training on Efficient Panoptic Segmentation](https://mlsanity.com/paper/2306.17319/) (2023)
- [1st Place Solution for PVUW Challenge 2023: Video Panoptic Segmentation](https://mlsanity.com/paper/2306.04091/) (2023)
- [A Simple Video Segmenter by Tracking Objects Along Axial Trajectories](https://mlsanity.com/paper/2311.18537/) (2023)

## Abstract (arXiv)

Video Panoptic Segmentation (VPS) is a challenging task that is extends from image panoptic segmentation.VPS aims to simultaneously classify, track, segment all objects in a video, including both things and stuff. Due to its wide application in many downstream tasks such as video understanding, video editing, and autonomous driving. In order to deal with the task of video panoptic segmentation in the wild, we propose a robust integrated video panoptic segmentation solution. We use DVIS++ framework as our baseline to generate the initial masks. Then,we add an additional image semantic segmentation model to further improve the performance of semantic classes.Finally, our method achieves state-of-the-art performance with a VPQ score of 56.36 and 57.12 in the development and test phases, respectively, and ultimately ranked 2nd in the VPS track of the PVUW Challenge at CVPR2024.