mlsanityData to 14 Oct 2026

Scene-Centric Unsupervised Video Panoptic Segmentation

arXiv 2606.04925 ↗First posted 3 Jun 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Video panoptic segmentation (VPS) aims to jointly detect, segment, and track all objects while partitioning the video into semantically consistent regions. We introduce the task setting of unsupervised VPS, omitting any human supervision. Existing unsupervised scene understanding works mainly focused on image segmentation tasks; the video domain remains underexplored. We propose VideoCUPS, the first unsupervised VPS approach. VideoCUPS generates temporally consistent panoptic video pseudo-labels from scene-centric videos by exploiting unsupervised depth, motion, and visual cues. Training on these pseudo-labels using a novel Video DropLoss yields an accurate, unsupervised VPS model. To benchmark progress, we introduce a comprehensive evaluation protocol and four competitive baselines, extending state-of-the-art unsupervised panoptic image and instance video segmentation models to VPS. VideoCUPS outperforms all baselines and demonstrates strong label-efficient learning. With VideoCUPS, our evaluation protocol, and baselines, we provide a strong foundation for future research on unsupervised VPS.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 107 of its 137 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

107 of 137 references.

References · 107

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2023Segment Anythingcited by 8,234
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  10. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  11. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  12. 2020Exploring Simple Siamese Representation Learningcited by 2,866
38 more
  1. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  5. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  6. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  7. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  8. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  9. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  10. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  11. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  12. 2016Simple Online and Realtime Trackingcited by 868
  13. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  14. 2018Panoptic Segmentationcited by 802
  15. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  16. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  17. 2019Panoptic Feature Pyramid Networkscited by 645
  18. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
  19. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  20. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  21. 2019Video Instance Segmentationcited by 393
  22. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  23. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  24. 2019Kornia: an Open Source Differentiable Computer Vision Library for PyTorchcited by 232
  25. 2019MOTS: Multi-Object Tracking and Segmentationcited by 223
  26. 2017Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detectioncited by 209
  27. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  28. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  29. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 187
  30. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 174
  31. 2021Mask2Former for Video Instance Segmentationcited by 171
  32. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  33. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  34. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  35. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 148
  36. 2020Video Panoptic Segmentationcited by 142
  37. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  38. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131

50 most cited of 107 All 107 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Generalist Open-World Temporal Perceptioncited by 0