mlsanityData to 7 Oct 2026

Exploiting Unlabeled Data with Vision and Language Models for Object Detection

arXiv 2207.08954 ↗First posted 18 Jul 2022cs.CVcited by 97

Not on the map. Why?

Cited by 24 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Building robust and generic object detection frameworks requires scaling to larger label spaces and bigger training datasets. However, it is prohibitively costly to acquire annotations for thousands of categories at a large scale. We propose a novel method that leverages the rich semantics available in recent vision and language models to localize and classify objects in unlabeled images, effectively generating pseudo labels for object detection. Starting with a generic and class-agnostic region proposal mechanism, we use vision and language models to categorize each region of an image into any object category that is required for downstream tasks. We demonstrate the value of the generated pseudo labels in two specific tasks, open-vocabulary detection, where a model needs to generalize to unseen object categories, and semi-supervised object detection, where additional unlabeled images can be used to improve the model. Our empirical evaluation shows the effectiveness of the pseudo labels in both tasks, where we outperform competitive baselines and achieve a novel state-of-the-art for open-vocabulary object detection. Our code is available at https://github.com/xiaofeng94/VL-PLM.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 51 of its 67 references that are in the map. First posted on arXiv 18 Jul 2022.

Builds on

51 of 67 references.

References · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2017Mask R-CNNcited by 7,259
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  11. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  12. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
38 more
  1. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  2. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  3. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  4. 2016Modeling Context in Referring Expressionscited by 1,054
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  6. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  7. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  8. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  9. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  10. 2022Language-driven Semantic Segmentationcited by 711
  11. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  12. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  13. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  14. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  15. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  16. 2018nocaps: novel object captioning at scalecited by 444
  17. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  18. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  19. 2017Embodied Question Answeringcited by 335
  20. 2021End-to-End Semi-Supervised Object Detection with Soft Teachercited by 315
  21. 2020A Simple Semi-Supervised Learning Framework for Object Detectioncited by 282
  22. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  23. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
  24. 2018Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptationcited by 218
  25. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  26. 2018Zero-Shot Object Detectioncited by 189
  27. 2015Learning Deep Structure-Preserving Image-Text Embeddingscited by 139
  28. 2017Data Distillation: Towards Omni-Supervised Learningcited by 126
  29. 2018Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answeringcited by 113
  30. 2019Learning by Abstraction: The Neural State Machinecited by 113
  31. 2021Instant-Teaching: An End-to-End Semi-Supervised Object Detection Frameworkcited by 112
  32. 2014From Captions to Visual Concepts and Backcited by 109
  33. 2021Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labelingcited by 84
  34. 2020Object Detection with a Unified Label Space from Multiple Datasetscited by 65
  35. 2019Dont Even Look Once: Synthesizing Features for Zero-Shot Detectioncited by 58
  36. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  37. 2020Boosting Weakly Supervised Object Detection with Progressive Knowledge Transfercited by 28
  38. 2021Boosting Weakly Supervised Object Detection via Learning Bounding Box Adjusterscited by 20

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 40; 2024: 28; 2025: 15; 2026: 11.

97 papers.

Sort:

By area:

  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  2. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  3. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  4. 2022F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Modelscited by 149
  5. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  6. 2022Learning Object-Language Alignments for Open-Vocabulary Object Detectioncited by 95
  7. 2023Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformerscited by 91
  8. 2023V3Det: Vast Vocabulary Visual Detection Datasetcited by 81
  9. 2023Fine-Grained Visual Promptingcited by 81
  10. 2023A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervisioncited by 78
  11. 2023Object-Aware Distillation Pyramid for Open-Vocabulary Object Detectioncited by 63
  12. 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Modelscited by 60
38 more
  1. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  2. 2024DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detectioncited by 48
  3. 2023EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignmentcited by 44
  4. 2024Cross-Domain Few-Shot Object Detection via Enhanced Open-Set Object Detectorcited by 43
  5. 2023CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Groundingcited by 35
  6. 2025R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuningcited by 33
  7. 2023Contrastive Feature Masking Open-Vocabulary Vision Transformercited by 31
  8. 2023LP-OVOD: Open-Vocabulary Object Detection by Linear Probingcited by 29
  9. 2022Open-vocabulary Attribute Detectioncited by 28
  10. 2024Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reductioncited by 28
  11. 2024InstaGen: Enhancing Object Detection by Training on Synthetic Datasetcited by 26
  12. 2024AIDE: An Automatic Data Engine for Object Detection in Autonomous Drivingcited by 26
  13. 2023Taming Self-Training for Open-Vocabulary Object Detectioncited by 25
  14. 2023MaMMUT: A Simple Architecture for Joint Learning for MultiModal Taskscited by 24
  15. 2024Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detectioncited by 24
  16. 2023Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learningcited by 22
  17. 2024OV-DQUO: Open-Vocabulary DETR with Denoising Text Query Training and Open-World Unknown Objects Supervisioncited by 22
  18. 2023Unified Adversarial Patch for Visible-Infrared Cross-modal Attacks in the Physical Worldcited by 21
  19. 2024Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environmentscited by 21
  20. 2024Retrieval-Augmented Open-Vocabulary Object Detectioncited by 20
  21. 2024CLIP-Driven Semantic Discovery Network for Visible-Infrared Person Re-Identificationcited by 19
  22. 2024LaMI-DETR: Open-Vocabulary Detection with Language Model Instructioncited by 19
  23. 2023MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detectioncited by 17
  24. 2023Mask-free OVIS: Open-Vocabulary Instance Segmentation without Manual Mask Annotationscited by 16
  25. 2023OmniLabel: A Challenging Benchmark for Language-Based Object Detectioncited by 16
  26. 2023ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detectioncited by 15
  27. 2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
  28. 2023Unified Visual Relationship Detection with Vision and Language Modelscited by 13
  29. 2023Detect Everything with Few Examplescited by 13
  30. 2023Generating Enhanced Negatives for Training Language-Based Object Detectorscited by 11
  31. 2024As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?cited by 11
  32. 2024MarvelOVD: Marrying Object Recognition and Vision-Language Models for Robust Open-Vocabulary Object Detectioncited by 11
  33. 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluationcited by 10
  34. 2025A Hierarchical Semantic Distillation Framework for Open-Vocabulary Object Detectioncited by 9
  35. 2023Mobile User Interface Element Detection Via Adaptively Prompt Tuningcited by 8
  36. 2023Region-centric Image-Language Pretraining for Open-Vocabulary Detectioncited by 8
  37. 2025LED: LLM Enhanced Open-Vocabulary Object Detection without Human Curated Data Generationcited by 8
  38. 2023Open-Vocabulary Object Detection via Scene Graph Discoverycited by 7

50 most cited of 97 All 97 (JSON)