mlsanityData to 14 Oct 2026

Cut and Learn for Unsupervised Object Detection and Instance Segmentation

arXiv 2301.11320 ↗First posted 26 Jan 2023cs.CV cs.AI cs.LGcited by 187

Not on the map. Why?

Abstract (arXiv)

We propose Cut-and-LEaRn (CutLER), a simple approach for training unsupervised object detection and segmentation models. We leverage the property of self-supervised models to 'discover' objects without supervision and amplify it to train a state-of-the-art localization model without any human labels. CutLER first uses our proposed MaskCut approach to generate coarse masks for multiple objects in an image and then learns a detector on these masks using our robust loss function. We further improve the performance by self-training the model on its predictions. Compared to prior work, CutLER is simpler, compatible with different detection architectures, and detects multiple objects. CutLER is also a zero-shot unsupervised detector and improves detection performance AP50 by over 2.7 times on 11 benchmarks across domains like video frames, paintings, sketches, etc. With finetuning, CutLER serves as a low-shot detector surpassing MoCo-v2 by 7.3% APbox and 6.6% APmask on COCO when training with 5% labels.

Key facts. Cited by 187 papers in the map (data through 14 Oct 2026). Builds on 45 of its 57 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

45 of 57 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2017Mask R-CNNcited by 7,277
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  12. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
33 more
  1. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  2. 2015Fast R-CNNcited by 2,933
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  4. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  9. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  10. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  12. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  13. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 767
  14. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
  15. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  16. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  17. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  18. 2015YFCC100M: The New Data in Multimedia Researchcited by 235
  19. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  20. 2018Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptationcited by 218
  21. 2017Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detectioncited by 209
  22. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  23. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  24. 2022TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cutcited by 113
  25. 2022FreeSOLO: Learning to Segment Objects without Annotationscited by 102
  26. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  27. 2021DETReg: Unsupervised Pretraining with Region Priors for Object Detectioncited by 76
  28. 2015Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposalscited by 69
  29. 2020Toward unsupervised, multi-object discovery in large-scale image collectionscited by 65
  30. 2020OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learningcited by 65
  31. 2021Large-Scale Unsupervised Object Discoverycited by 54
  32. 2022Discovering Object Masks with Transformers for Unsupervised Semantic Segmentationcited by 54
  33. 2020Unsupervised Feature Learning by Cross-Level Instance-Group Discriminationcited by 51

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 48; 2024: 63; 2025: 49; 2026: 27.

187 papers.

Sort:

By area:

  1. 2023Vision Transformers Need Registerscited by 871
  2. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  3. 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 96
  4. 2023Object-Centric Slot Diffusioncited by 90
  5. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 64
  6. 2023Zero-shot Referring Image Segmentation with Global-Local Context Featurescited by 61
  7. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  8. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  9. 2024Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary Segmentationcited by 44
  10. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 43
  11. 2023CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Freecited by 39
  12. 2024Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentationcited by 39
38 more
  1. 2024In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentationcited by 38
  2. 2023VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentationcited by 37
  3. 2023Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representationscited by 35
  4. 2023Unsupervised Universal Image Segmentationcited by 35
  5. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  6. 2024Adaptive Slot Attention: Object Discovery with Dynamic Slot Numbercited by 34
  7. 2023UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenescited by 31
  8. 2024Segment Anything without Supervisioncited by 28
  9. 2024DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cutcited by 26
  10. 2023Emergence of Segmentation with Minimalistic White-Box Transformerscited by 24
  11. 2023LEOD: Label-Efficient Object Detection for Event Camerascited by 24
  12. 2024Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentationcited by 24
  13. 2025Seg4Diff: Unveiling Open-Vocabulary Segmentation in Text-to-Image Diffusion Transformerscited by 24
  14. 2023Component-aware anomaly detection framework for adjustable and logical industrial visual inspectioncited by 22
  15. 2024UNION: Unsupervised 3D Object Detection using Object Appearance-based Pseudo-Classescited by 22
  16. 2025Simplifying DINO via Coding Rate Regularizationcited by 21
  17. 2025SAMRefiner: Taming Segment Anything Model for Universal Mask Refinementcited by 20
  18. 2023Few-Shot Panoptic Segmentation With Foundation Modelscited by 18
  19. 2024A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpaintingcited by 18
  20. 2023Object-centric Learning with Cyclic Walks between Parts and Wholecited by 16
  21. 2023Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Surveycited by 16
  22. 2024CuVLER: Enhanced Unsupervised Object Discoveries through Exhaustive Self-Supervised Transformerscited by 16
  23. 2024ConceptExpress: Harnessing Diffusion Models for Single-image Unsupervised Concept Extractioncited by 16
  24. 2023Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Modelscited by 15
  25. 2024Can Visual Foundation Models Achieve Long-term Point Tracking?cited by 14
  26. 2024Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentationcited by 13
  27. 2025Feed-Forward SceneDINO for Unsupervised Semantic Scene Completioncited by 13
  28. 2023Weakly-supervised Contrastive Learning for Unsupervised Object Discoverycited by 12
  29. 2024Open-Vocabulary Segmentation with Unpaired Mask-Text Supervisioncited by 12
  30. 2024Part2Object: Hierarchical Unsupervised 3D Instance Segmentationcited by 12
  31. 2024SANGRIA: Surgical Video Scene Graph Optimization for Surgical Workflow Predictioncited by 12
  32. 2023Object-Centric Learning with Slot Mixture Modulecited by 11
  33. 2023Understanding Physical Dynamics with Counterfactual World Modelingcited by 11
  34. 2024DiPEx: Dispersing Prompt Expansion for Class-Agnostic Object Detectioncited by 11
  35. 2024VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentationcited by 11
  36. 2023Understanding Masked Autoencoders From a Local Contrastive Perspectivecited by 10
  37. 2023Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentationcited by 10
  38. 2023HEAP: Unsupervised Object Discovery and Localization with Contrastive Groupingcited by 10

50 most cited of 187 All 187 (JSON)