mlsanityData to 14 Oct 2026

Zero-shot Unsupervised Transfer Instance Segmentation

arXiv 2304.14376 ↗First posted 27 Apr 2023cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Segmentation is a core computer vision competency, with applications spanning a broad range of scientifically and economically valuable domains. To date, however, the prohibitive cost of annotation has limited the deployment of flexible segmentation models. In this work, we propose Zero-shot Unsupervised Transfer Instance Segmentation (ZUTIS), a framework that aims to meet this challenge. The key strengths of ZUTIS are: (i) no requirement for instance-level or pixel-level annotations; (ii) an ability of zero-shot transfer, i.e., no assumption on access to a target data distribution; (iii) a unified framework for semantic and instance segmentations with solid performance on both tasks compared to state-of-the-art unsupervised methods. While comparing to previous work, we show ZUTIS achieves a gain of 2.2 mask AP on COCO-20K and 14.5 mIoU on ImageNet-S with 919 categories for instance and semantic segmentations, respectively. The code is made publicly available.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 56 of its 77 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

56 of 77 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2017Mask R-CNNcited by 7,277
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
38 more
  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  2. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  3. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  4. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  5. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  6. 2022Language-driven Semantic Segmentationcited by 717
  7. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
  8. 2021Extract Free Dense Labels from CLIPcited by 583
  9. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
  10. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  11. 2021Image Segmentation Using Text and Image Promptscited by 512
  12. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 509
  13. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  14. 2021Intriguing Properties of Vision Transformerscited by 446
  15. 2019Zero-Shot Semantic Segmentationcited by 245
  16. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  17. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  18. 2021K-Net: Towards Unified Image Segmentationcited by 212
  19. 2021Decoupling Zero-Shot Semantic Segmentationcited by 199
  20. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  21. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 187
  22. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 167
  23. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  24. 2018Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Modelscited by 119
  25. 2022ReCo: Retrieve and Co-segment for Zero-shot Transfercited by 117
  26. 2022FreeSOLO: Learning to Segment Objects without Annotationscited by 102
  27. 2021Large-scale Unsupervised Semantic Segmentationcited by 101
  28. 2020Context-aware Feature Generation for Zero-shot Semantic Segmentationcited by 94
  29. 2021Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labelingcited by 84
  30. 2017Open Vocabulary Scene Parsingcited by 76
  31. 2017Learning to Segment Every Thingcited by 65
  32. 2020Toward unsupervised, multi-object discovery in large-scale image collectionscited by 65
  33. 2020Self-Supervised Visual Representation Learning from Hierarchical Groupingcited by 63
  34. 2020Autoregressive Unsupervised Image Segmentationcited by 60
  35. 2021A Closer Look at Self-training for Zero-Label Semantic Segmentationcited by 57
  36. 2022Unsupervised Salient Object Detection with Spectral Cluster Votingcited by 56
  37. 2020Learning to Segment the Tailcited by 54
  38. 2022Discovering Object Masks with Transformers for Unsupervised Semantic Segmentationcited by 54

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 0; 2026: 1.

4 papers.

  1. 2024Zero-Shot Automatic Annotation and Instance Segmentation using LLM-Generated Datasets: Eliminating Field Imaging and Manual Annotation for Deep Learning Model Developmentcited by 5
  2. 2023Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignmentcited by 4
  3. 2024Language-Guided Instance-Aware Domain-Adaptive Panoptic Segmentationcited by 4
  4. 2026Cognitive Twins: Investigating Personalized Thinking Model Building and Its Performance Enhancement with Human-in-the-Loopcited by 0