mlsanityData to 7 Oct 2026

1st Place Winner of the 2024 Pixel-level Video Understanding in the Wild (CVPR'24 PVUW) Challenge in Video Panoptic Segmentation and Best Long Video Consistency of Video Semantic Segmentation

arXiv 2406.05352 ↗First posted 8 Jun 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The third Pixel-level Video Understanding in the Wild (PVUW CVPR 2024) challenge aims to advance the state of art in video understanding through benchmarking Video Panoptic Segmentation (VPS) and Video Semantic Segmentation (VSS) on challenging videos and scenes introduced in the large-scale Video Panoptic Segmentation in the Wild (VIPSeg) test set and the large-scale Video Scene Parsing in the Wild (VSPW) test set, respectively. This paper details our research work that achieved the 1st place winner in the PVUW'24 VPS challenge, establishing state of art results in all metrics, including the Video Panoptic Quality (VPQ) and Segmentation and Tracking Quality (STQ). With minor fine-tuning our approach also achieved the 3rd place in the PVUW'24 VSS challenge ranked by the mIoU (mean intersection over union) metric and the first place ranked by the VC16 (16-frame video consistency) metric. Our winning solution stands on the shoulders of giant foundational vision transformer model (DINOv2 ViT-g) and proven multi-stage Decoupled Video Instance Segmentation (DVIS) frameworks for video understanding.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 8 Jun 2024.

Builds on

16 of 22 references.

References · 16

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  5. 2019Panoptic-DeepLabcited by 362
  6. 2021K-Net: Towards Unified Image Segmentationcited by 211
  7. 2022Separable Self-attention for Mobile Vision Transformerscited by 163
  8. 2020Video Panoptic Segmentationcited by 141
  9. 2020ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentationcited by 95
  10. 2022MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Trainingcited by 94
  11. 2022Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentationcited by 79
  12. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
4 more
  1. 2024ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictionscited by 65
  2. 2023DVIS++: Improved Decoupled Framework for Universal Video Segmentationcited by 46
  3. 2023TarViS: A Unified Approach for Target-based Video Segmentationcited by 32
  4. 2023Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentationcited by 24

Built on by

Not cited yet.