mlsanityData to 14 Oct 2026

Localized Vision-Language Matching for Open-vocabulary Object Detection

arXiv 2205.06160 ↗First posted 12 May 2022cs.CV cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

In this work, we propose an open-vocabulary object detection method that, based on image-caption pairs, learns to detect novel object classes along with a given set of known classes. It is a two-stage training approach that first uses a location-guided image-caption matching technique to learn class labels for both novel and known classes in a weakly-supervised manner and second specializes the model for the object detection task using known class annotations. We show that a simple language model fits better than a large contextualized language model for detecting novel objects. Moreover, we introduce a consistency-regularization technique to better exploit image-caption pair information. Our method compares favorably to existing open-vocabulary detection approaches while being data-efficient. Source code is available at https://github.com/lmb-freiburg/locov .

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 12 May 2022.

Builds on

35 of 47 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  7. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  8. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  9. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  10. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  11. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  12. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
23 more
  1. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  2. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  3. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  4. 2021Unbiased Teacher for Semi-Supervised Object Detectioncited by 306
  5. 2020Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformerscited by 288
  6. 2020A Simple Semi-Supervised Learning Framework for Object Detectioncited by 282
  7. 2018Zero-Shot Object Detectioncited by 189
  8. 2015Weakly Supervised Deep Detection Networkscited by 183
  9. 2017Learning Two-Branch Neural Networks for Image-Text Matching Taskscited by 119
  10. 2020Where Does It Exist: Spatio-Temporal Video Grounding for Multi-Form Sentencescited by 118
  11. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
  12. 2020COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learningcited by 100
  13. 2021Learning to Predict Visual Attributes in the Wildcited by 96
  14. 2021Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformerscited by 93
  15. 2018Dual Encoding for Zero-Example Video Retrievalcited by 85
  16. 2021Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labelingcited by 84
  17. 2020Instance-aware, Context-focused, and Memory-efficient Weakly Supervised Object Detectioncited by 80
  18. 2019Weakly-Supervised Spatio-Temporally Grounding Natural Sentence in Videocited by 60
  19. 2019Dont Even Look Once: Synthesizing Features for Zero-Shot Detectioncited by 58
  20. 2015Weakly Supervised Object Localization with Multi-fold Multiple Instance Learningcited by 39
  21. 2020Video Object Grounding using Semantic Roles in Language Descriptioncited by 25
  22. 2020COBE: Contextualized Object Embeddings from Narrated Instructional Videocited by 12
  23. 2019Object-Aware Instance Labeling for Weakly Supervised Object Detectioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 4; 2025: 0; 2026: 2.

17 papers.

Sort:

By area:

  1. 2023Towards Open Vocabulary Learning: A Surveycited by 166
  2. 2023Object-Aware Distillation Pyramid for Open-Vocabulary Object Detectioncited by 63
  3. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 61
  4. 2023EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignmentcited by 44
  5. 2023OvarNet: Towards Open-vocabulary Object Attribute Recognitioncited by 43
  6. 2022Open-vocabulary Attribute Detectioncited by 29
  7. 2023LP-OVOD: Open-Vocabulary Object Detection by Linear Probingcited by 29
  8. 2024AIDE: An Automatic Data Engine for Object Detection in Autonomous Drivingcited by 26
  9. 2024T-VSL: Text-Guided Visual Sound Source Localization in Mixturescited by 19
  10. 2023Mask-free OVIS: Open-Vocabulary Instance Segmentation without Manual Mask Annotationscited by 16
  11. 2024MarvelOVD: Marrying Object Recognition and Vision-Language Models for Robust Open-Vocabulary Object Detectioncited by 11
  12. 2022Open Vocabulary Multi-Label Classification with Dual-Modal Decoder on Aligned Visual-Textual Featurescited by 6
5 more
  1. 2024Open-Vocabulary Object Detectors: Robustness Challenges under Distribution Shiftscited by 4
  2. 2023LOWA: Localize Objects in the Wild with Attributescited by 1
  3. 2023DST-Det: Simple Dynamic Self-Training for Open-Vocabulary Object Detectioncited by 1
  4. 2026Unveiling the Unknown: Open Vocabulary Object Detection with Scene Graphscited by 0
  5. 2026Open-Vocabulary Gaze Object Prediction: Benchmark and Methodcited by 0