mlsanityData to 7 Oct 2026

VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation

arXiv 2308.14710 ↗First posted 28 Aug 2023cs.CV cs.AI cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Existing approaches to unsupervised video instance segmentation typically rely on motion estimates and experience difficulties tracking small or divergent motions. We present VideoCutLER, a simple method for unsupervised multi-instance video segmentation without using motion-based learning signals like optical flow or training on natural videos. Our key insight is that using high-quality pseudo masks and a simple video synthesis method for model training is surprisingly sufficient to enable the resulting video model to effectively segment and track multiple instances across video frames. We show the first competitive unsupervised learning results on the challenging YouTubeVIS-2019 benchmark, achieving 50.7% APvideo^50 , surpassing the previous state-of-the-art by a large margin. VideoCutLER can also serve as a strong pretrained model for supervised video instance segmentation tasks, exceeding DINO by 15.9% on YouTubeVIS-2019 in terms of APvideo.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 28 Aug 2023.

Builds on

33 of 43 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2017Mask R-CNNcited by 7,259
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
21 more
  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  2. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  3. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  4. 2019Video Instance Segmentationcited by 391
  5. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  6. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
  7. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 186
  8. 2021Mask2Former for Video Instance Segmentationcited by 169
  9. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  10. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  11. 2022TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cutcited by 111
  12. 2022FreeSOLO: Learning to Segment Objects without Annotationscited by 102
  13. 2019RVOS: End-to-End Recurrent Network for Video Object Segmentationcited by 88
  14. 2019Joint-task Self-supervised Learning for Temporal Correspondencecited by 80
  15. 2019Unsupervised Moving Object Detection via Contextual Information Separationcited by 72
  16. 2022Segmenting Moving Objects via an Object-Centric Layered Representationcited by 63
  17. 2020Motion-Attentive Transition for Zero-Shot Video Object Segmentationcited by 62
  18. 2020Unsupervised Feature Learning by Cross-Level Instance-Group Discriminationcited by 51
  19. 2020UnOVOST: Unsupervised Offline Video Object Segmentation and Trackingcited by 34
  20. 2022Unsupervised Video Object Segmentation via Prototype Memory Networkcited by 30
  21. 2018Unsupervised Video Object Segmentation using Motion Saliency-Guided Spatio-Temporal Propagationcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 17; 2026: 8.

36 papers.

Sort:

By area:

  1. 2024Moving Object Segmentation: All You Need Is SAM (and Flow)cited by 30
  2. 2024Segment Anything without Supervisioncited by 27
  3. 2024RoMo: Robust Motion Segmentation Improves Structure from Motioncited by 20
  4. 2023Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Surveycited by 16
  5. 2024Can Visual Foundation Models Achieve Long-term Point Tracking?cited by 14
  6. 2025Learning segmentation from point trajectoriescited by 14
  7. 2024SANGRIA: Surgical Video Scene Graph Optimization for Surgical Workflow Predictioncited by 12
  8. 2023Appearance-Based Refinement for Object-Centric Motion Segmentationcited by 11
  9. 2025Vector-Quantized Vision Foundation Models for Object-Centric Learningcited by 11
  10. 2023Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentationcited by 10
  11. 2025Scene-Centric Unsupervised Panoptic Segmentationcited by 10
  12. 2024Rethinking Image-to-Video Adaptation: An Object-centric Perspectivecited by 8
24 more
  1. 2025HiERO: understanding the hierarchy of human behavior enhances reasoning on egocentric videoscited by 7
  2. 2025Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognitioncited by 7
  3. 2024Multi-Scale Fusion for Object Representationcited by 6
  4. 2025Motion-Refined DINOSAUR for Unsupervised Multi-Object Discoverycited by 4
  5. 2026WildRayZer: Self-supervised Large View Synthesis in Dynamic Environmentscited by 4
  6. 2026PyraTok: Language-Aligned Pyramidal Tokenizer for Video Understanding and Generationcited by 4
  7. 2025TRACE: Your Diffusion Model is Secretly an Instance Edge Detectorcited by 3
  8. 2025FlowCut: Unsupervised Video Instance Segmentation via Temporal Mask Matchingcited by 2
  9. 2025A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospectscited by 2
  10. 2025Hierarchical Visual Prompt Learning for Continual Video Instance Segmentationcited by 2
  11. 2025Unsupervised Part Discovery via Descriptor-Based Masked Image Restoration with Optimized Constraintscited by 1
  12. 2025AutoQ-VIS: Improving Unsupervised Video Instance Segmentation via Automatic Quality Assessmentcited by 1
  13. 2026GMOS: Grounding Moving Object Segmentation in 3D Space and Timecited by 1
  14. 2026Scene-Centric Unsupervised Video Panoptic Segmentationcited by 1
  15. 2024Improving Weakly-supervised Video Instance Segmentation by Leveraging Spatio-temporal Consistencycited by 0
  16. 2025Unsupervised Online 3D Instance Segmentation with Synthetic Sequences and Dynamic Losscited by 0
  17. 2025MapSAM2: Adapting SAM2 for Automatic Segmentation of Historical Map Images and Time Seriescited by 0
  18. 2025The Dynamic Prior: Understanding 3D Structures for Casual Dynamic Videoscited by 0
  19. 2025Boosting Unsupervised Video Instance Segmentation with Automatic Quality-Guided Self-Trainingcited by 0
  20. 2025S2D: Sparse-To-Dense Keymask Distillation for Unsupervised Video Instance Segmentationcited by 0
  21. 2026Spatial-Regularization-Aware Dual-Branch Collaborative Inference for Training-Free OVSS in Remote Sensing Imagerycited by 0
  22. 2026VVitCutLER: Towards Unsupervised Object Detection and Segmentation in Videoscited by 0
  23. 2026Spectral-Target Physical Latent Structuring for JEPA-Style World Modelscited by 0
  24. 2026Seeing as Humans Do: Learning from Motion to Segment Anything Without Supervisioncited by 0