mlsanityData to 7 Oct 2026

Open Vocabulary Object Detection with Pseudo Bounding-Box Labels

arXiv 2111.09452 ↗First posted 18 Nov 2021cs.CVcited by 71

Not on the map. Why?

Abstract (arXiv)

Despite great progress in object detection, most existing methods work only on a limited set of object categories, due to the tremendous human effort needed for bounding-box annotations of training data. To alleviate the problem, recent open vocabulary and zero-shot detection methods attempt to detect novel object categories beyond those seen during training. They achieve this goal by training on a pre-defined base categories to induce generalization to novel objects. However, their potential is still constrained by the small set of base categories available for training. To enlarge the set of base classes, we propose a method to automatically generate pseudo bounding-box annotations of diverse objects from large-scale image-caption pairs. Our method leverages the localization ability of pre-trained vision-language models to generate pseudo bounding-box labels and then directly uses them for training object detectors. Experimental results show that our method outperforms the state-of-the-art open vocabulary detector by 8% AP on COCO novel categories, by 6.3% AP on PASCAL VOC, by 2.3% AP on Objects365 and by 2.8% AP on LVIS. Code is available at https://github.com/salesforce/PB-OVD.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 18 Nov 2021.

Builds on

26 of 36 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2017Mask R-CNNcited by 7,259
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2015Fast R-CNNcited by 2,930
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  10. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  11. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  12. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
14 more
  1. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  2. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  3. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  4. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  5. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  6. 2021Towards Open World Object Detectioncited by 297
  7. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  8. 2018Zero-Shot Object Detectioncited by 189
  9. 2015Weakly Supervised Deep Detection Networkscited by 183
  10. 2018PCL: Proposal Cluster Learning for Weakly Supervised Object Detectioncited by 97
  11. 2019Dont Even Look Once: Synthesizing Features for Zero-Shot Detectioncited by 58
  12. 2019Cap2Det: Learning to Amplify Weak Caption Supervision for Object Detectioncited by 31
  13. 2018Zero-Shot Detectioncited by 27
  14. 2017C-WSL: Count-guided Weakly Supervised Localizationcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 36; 2024: 17; 2025: 9; 2026: 4.

71 papers.

Sort:

By area:

  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  2. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
  3. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  4. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  5. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  6. 2022GRiT: A Generative Region-to-text Transformer for Object Understandingcited by 119
  7. 2023Aligning Bag of Regions for Open-Vocabulary Object Detectioncited by 119
  8. 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 114
  9. 2023V3Det: Vast Vocabulary Visual Detection Datasetcited by 81
  10. 2023Object-Aware Distillation Pyramid for Open-Vocabulary Object Detectioncited by 63
  11. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  12. 2024DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detectioncited by 48
38 more
  1. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  2. 2024OpenESS: Event-based Semantic Scene Understanding with Open Vocabulariescited by 37
  3. 2023MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentationcited by 35
  4. 2023LP-OVOD: Open-Vocabulary Object Detection by Linear Probingcited by 29
  5. 2022Open-vocabulary Attribute Detectioncited by 28
  6. 2023Zero-shot Visual Relation Detection via Composite Visual Cues from Large Language Modelscited by 27
  7. 2024InstaGen: Enhancing Object Detection by Training on Synthetic Datasetcited by 26
  8. 2023Taming Self-Training for Open-Vocabulary Object Detectioncited by 25
  9. 2024Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detectioncited by 24
  10. 2022Learning to Detect and Segment for Open Vocabulary Object Detectioncited by 22
  11. 2023CLIM: Contrastive Language-Image Mosaic for Region Representationcited by 21
  12. 2024Retrieval-Augmented Open-Vocabulary Object Detectioncited by 20
  13. 2024ImOV3D: Learning Open-Vocabulary Point Clouds 3D Object Detection from Only 2D Imagescited by 17
  14. 2023Mask-free OVIS: Open-Vocabulary Instance Segmentation without Manual Mask Annotationscited by 16
  15. 2024Scene Graph Generation with Role-Playing Large Language Modelscited by 16
  16. 2023ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detectioncited by 15
  17. 2023Open-Vocabulary Object Detection using Pseudo Caption Labelscited by 13
  18. 2024Open World Object Detection: A Surveycited by 13
  19. 2023Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentationcited by 12
  20. 2025Refining CLIP's Spatial Awareness: A Visual-Centric Perspectivecited by 12
  21. 2023Weakly Supervised Open-Vocabulary Object Detectioncited by 11
  22. 2024MarvelOVD: Marrying Object Recognition and Vision-Language Models for Robust Open-Vocabulary Object Detectioncited by 11
  23. 2024Cross-Modality Gait Recognition: Bridging LiDAR and Camera Modalities for Human Identificationcited by 10
  24. 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluationcited by 10
  25. 2023CLIP Is Also a Good Teacher: A New Learning Framework for Inductive Zero-shot Semantic Segmentationcited by 9
  26. 2025A Hierarchical Semantic Distillation Framework for Open-Vocabulary Object Detectioncited by 9
  27. 2023Region-centric Image-Language Pretraining for Open-Vocabulary Detectioncited by 8
  28. 2023Open-Vocabulary Object Detection via Scene Graph Discoverycited by 7
  29. 2023Exploring Multi-Modal Contextual Knowledge for Open-Vocabulary Object Detectioncited by 7
  30. 2023Rethinking Evaluation Metrics of Open-Vocabulary Segmentaioncited by 7
  31. 2022Open Vocabulary Multi-Label Classification with Dual-Modal Decoder on Aligned Visual-Textual Featurescited by 6
  32. 2025UniGS: Unified Language-Image-3D Pretraining with Gaussian Splattingcited by 5
  33. 2023Boosting Weakly Supervised Object Detection using Fusion and Priors from Hallucinated Depthcited by 4
  34. 2023Improving Reference-based Distinctive Image Captioning with Contrastive Rewardscited by 4
  35. 2023Unified Open-Vocabulary Dense Visual Predictioncited by 4
  36. 2023Catch Me If You Can Describe Me: Open-Vocabulary Camouflaged Instance Segmentation with Diffusioncited by 3
  37. 2024OVMR: Open-Vocabulary Recognition with Multi-Modal Referencescited by 3
  38. 2024End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Promptingcited by 3

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Scaling Open-Vocabulary Object Detection
  2. 2023EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignment
  3. 2022Open-Vocabulary DETR with Conditional Matching
  4. 2024Sampling Bag of Views for Open-Vocabulary Object Detection
  5. 2024LLMs Meet VLMs: Boost Open Vocabulary Object Detection with Fine-grained Descriptors
  6. 2025Open-Det: An Efficient Learning Framework for Open-Ended Detection
  7. 2026NoOVD: Novel Category Discovery and Embedding for Open-Vocabulary Object Detection
  8. 2023Three ways to improve feature alignment for open vocabulary detection