mlsanityData to 14 Oct 2026

ReCo: Retrieve and Co-segment for Zero-shot Transfer

arXiv 2206.07045 ↗First posted 14 Jun 2022cs.CV cs.AI cs.LGcited by 117

Not on the map. Why?

Cited by 22 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Semantic segmentation has a broad range of applications, but its real-world impact has been significantly limited by the prohibitive annotation costs necessary to enable deployment. Segmentation methods that forgo supervision can side-step these costs, but exhibit the inconvenient requirement to provide labelled examples from the target distribution to assign concept names to predictions. An alternative line of work in language-image pre-training has recently demonstrated the potential to produce models that can both assign names across large vocabularies of concepts and enable zero-shot transfer for classification, but do not demonstrate commensurate segmentation abilities. In this work, we strive to achieve a synthesis of these two approaches that combines their strengths. We leverage the retrieval abilities of one such language-image pre-trained model, CLIP, to dynamically curate training sets from unlabelled images for arbitrary collections of concept names, and leverage the robust correspondences offered by modern image representations to co-segment entities among the resulting collections. The synthetic segment collections are then employed to construct a segmentation model (without requiring pixel labels) whose knowledge of concepts is inherited from the scalable pre-training process of CLIP. We demonstrate that our approach, termed Retrieve and Co-segment (ReCo) performs favourably to unsupervised segmentation approaches while inheriting the convenience of nameable predictions and zero-shot transfer. We also demonstrate ReCo's ability to generate specialist segmenters for extremely rare objects.

Key facts. Cited by 117 papers in the map (data through 14 Oct 2026). Builds on 70 of its 110 references that are in the map. First posted on arXiv 14 Jun 2022.

Builds on

70 of 110 references.

References · 70

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
38 more
  1. 2017Billion-scale similarity search with GPUscited by 2,497
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  3. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  4. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  5. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  6. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  7. 2022Language-driven Semantic Segmentationcited by 717
  8. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  9. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
  10. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
  11. 2021Image Segmentation Using Text and Image Promptscited by 512
  12. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  13. 2021Intriguing Properties of Vision Transformerscited by 446
  14. 2020Semi-Supervised Semantic Segmentation with Cross-Consistency Trainingcited by 376
  15. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 331
  16. 2018Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentationcited by 321
  17. 2015What's the Point: Semantic Segmentation with Point Supervisioncited by 298
  18. 2019Zero-Shot Semantic Segmentationcited by 245
  19. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  20. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  21. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  22. 2016Simple Does It: Weakly Supervised Instance and Semantic Segmentationcited by 227
  23. 2021Decoupling Zero-Shot Semantic Segmentationcited by 199
  24. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  25. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  26. 2015ParseNet: Looking Wider to See Bettercited by 177
  27. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 167
  28. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  29. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  30. 2018Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentationcited by 145
  31. 2020Weakly-Supervised Semantic Segmentation via Sub-category Explorationcited by 145
  32. 2017Deep Extreme Cut: From Extreme Points to Object Segmentationcited by 127
  33. 2020CRNet: Cross-Reference Networks for Few-Shot Segmentationcited by 101
  34. 2015Constrained Convolutional Neural Networks for Weakly Supervised Segmentationcited by 100
  35. 2019Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentationcited by 98
  36. 2019SegSort: Segmentation by Discriminative Sorting of Segmentscited by 96
  37. 2020Context-aware Feature Generation for Zero-shot Semantic Segmentationcited by 94
  38. 2021Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labelingcited by 84

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 29; 2024: 37; 2025: 28; 2026: 18.

117 papers.

Sort:

By area:

  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  2. 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 170
  3. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  4. 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
  5. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  6. 2022SuS-X: Training-Free Name-Only Transfer of Vision-Language Modelscited by 119
  7. 2024ClearCLIP: Decomposing CLIP Representations for Dense Vision-Language Inferencecited by 113
  8. 2024ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentationcited by 111
  9. 2023Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervisioncited by 99
  10. 2023A Closer Look at the Explainability of Contrastive Language-Image Pre-trainingcited by 96
  11. 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 96
  12. 2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
38 more
  1. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  2. 2024Pay Attention to Your Neighbours: Training-Free Open-Vocabulary Semantic Segmentationcited by 77
  3. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
  4. 2023CLIP as RNN: Segment Countless Visual Concepts without Training Endeavorcited by 69
  5. 2023SILC: Improving Vision Language Pretraining with Self-Distillationcited by 66
  6. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 64
  7. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 61
  8. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  9. 2024Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generationcited by 44
  10. 2024Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary Segmentationcited by 44
  11. 2022ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentationcited by 42
  12. 2023SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidancecited by 41
  13. 2023CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Freecited by 39
  14. 2024Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentationcited by 39
  15. 2024In Defense of Lazy Visual Grounding for Open-Vocabulary Semantic Segmentationcited by 38
  16. 2023Exposing and Mitigating Spurious Correlations for Cross-Modal Retrievalcited by 36
  17. 2023CLIP-S4: Language-Guided Self-Supervised Semantic Segmentationcited by 34
  18. 2024CorrCLIP: Reconstructing Patch Correlations in CLIP for Open-Vocabulary Semantic Segmentationcited by 34
  19. 2024Self-Calibrated CLIP for Training-Free Open-Vocabulary Segmentationcited by 34
  20. 2025R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuningcited by 33
  21. 2025DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perceptioncited by 33
  22. 2024ResCLIP: Residual Attention for Training-free Dense Vision-language Inferencecited by 29
  23. 2023Uncovering Prototypical Knowledge for Weakly Open-Vocabulary Semantic Segmentationcited by 27
  24. 2024DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cutcited by 26
  25. 2025IteRPrimE: Zero-shot Referring Image Segmentation with Iterative Grad-CAM Refinement and Primary Word Emphasiscited by 26
  26. 2024SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Imagescited by 25
  27. 2024GiT: Towards Generalist Vision Transformer through Universal Language Interfacecited by 24
  28. 2024Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentationcited by 24
  29. 2025Seg4Diff: Unveiling Open-Vocabulary Segmentation in Text-to-Image Diffusion Transformerscited by 24
  30. 2023Rewrite Caption Semantics: Bridging Semantic Gaps for Language-Supervised Semantic Segmentationcited by 21
  31. 2024Image-Text Co-Decomposition for Text-Supervised Semantic Segmentationcited by 21
  32. 2022NamedMask: Distilling Segmenters from Complementary Foundation Modelscited by 20
  33. 2024kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabulariescited by 20
  34. 2025SAMRefiner: Taming Segment Anything Model for Universal Mask Refinementcited by 20
  35. 2023TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Trainingcited by 19
  36. 2023Causal Unsupervised Semantic Segmentationcited by 18
  37. 2024Cascade-CLIP: Cascaded Vision-Language Embeddings Alignment for Zero-Shot Semantic Segmentationcited by 17
  38. 2024CLIPer: Hierarchically Improving Spatial Representation of CLIP for Open-Vocabulary Semantic Segmentationcited by 17

50 most cited of 117 All 117 (JSON)