mlsanityData to 7 Oct 2026

PromptDet: Towards Open-vocabulary Detection using Uncurated Images

arXiv 2203.16513 ↗First posted 30 Mar 2022cs.CVcited by 133

Not on the map. Why?

Cited by 47 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The goal of this work is to establish a scalable pipeline for expanding an object detector towards novel/unseen categories, using zero manual annotations. To achieve that, we make the following four contributions: (i) in pursuit of generalisation, we propose a two-stage open-vocabulary object detector, where the class-agnostic object proposals are classified with a text encoder from pre-trained visual-language model; (ii) To pair the visual latent space (of RPN box proposals) with that of the pre-trained text encoder, we propose the idea of regional prompt learning to align the textual embedding space with regional visual object features; (iii) To scale up the learning procedure towards detecting a wider spectrum of objects, we exploit the available online resource via a novel self-training framework, which allows to train the proposed detector on a large corpus of noisy uncurated web images. Lastly, (iv) to evaluate our proposed detector, termed as PromptDet, we conduct extensive experiments on the challenging LVIS and MS-COCO dataset. PromptDet shows superior performance over existing approaches with fewer additional training images and zero manual annotations whatsoever. Project page with code: https://fcjian.github.io/promptdet.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 30 Mar 2022.

Builds on

24 of 37 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2017Mask R-CNNcited by 7,259
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  7. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  8. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  9. 2016YOLO9000: Better, Faster, Strongercited by 1,429
  10. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  11. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  12. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
12 more
  1. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  2. 2018Few-shot Object Detection via Feature Reweightingcited by 265
  3. 2021TOOD: Task-aligned One-stage Object Detectioncited by 218
  4. 2019Few-Shot Object Detection with Attention-RPN and Multi-Relation Detectorcited by 206
  5. 2018Zero-Shot Object Detectioncited by 189
  6. 2015Weakly Supervised Deep Detection Networkscited by 183
  7. 2017Open Vocabulary Scene Parsingcited by 76
  8. 2021Exploring Classification Equilibrium in Long-Tailed Object Detectioncited by 51
  9. 2017Link the head to the "beak": Zero Shot Learning from Noisy Text Description at Part Precisioncited by 46
  10. 2021Label, Verify, Correct: A Simple Few Shot Object Detection Methodcited by 43
  11. 2016Multi-Cue Zero-Shot Learning with Strong Supervisioncited by 22
  12. 2020Representation Sharing for Fast Object Detector Search and Beyondcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 47; 2024: 41; 2025: 27; 2026: 12.

133 papers.

Sort:

By area:

  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  2. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  3. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  4. 2023RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Modelcited by 180
  5. 2022PLA: Language-Driven Open-Vocabulary 3D Scene Understandingcited by 176
  6. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  7. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  8. 2023RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understandingcited by 93
  9. 2023Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformerscited by 91
  10. 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
  11. 2023V3Det: Vast Vocabulary Visual Detection Datasetcited by 81
  12. 2023Fine-Grained Visual Promptingcited by 81
38 more
  1. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  2. 2023Detecting Everything in the Open World: Towards Universal Object Detectioncited by 62
  3. 2023HOICLIP: Efficient Knowledge Transfer for HOI Detection with Vision-Language Modelscited by 60
  4. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  5. 2023Read-only Prompt Optimization for Vision-Language Few-shot Learningcited by 58
  6. 2023Multi-Modal Classifiers for Open-Vocabulary Object Detectioncited by 53
  7. 2022Bayesian Prompt Learning for Image-Language Model Generalizationcited by 51
  8. 2023CoDA: Collaborative Novel Box Discovery and Cross-modal Alignment for Open-vocabulary 3D Object Detectioncited by 51
  9. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  10. 2023EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignmentcited by 44
  11. 2022Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Modelscited by 40
  12. 2023Open-Vocabulary Semantic Segmentation with Decoupled One-Pass Networkcited by 40
  13. 2024LLMs Meet VLMs: Boost Open Vocabulary Object Detection with Fine-grained Descriptorscited by 39
  14. 2023Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognitioncited by 35
  15. 2023Contrastive Feature Masking Open-Vocabulary Vision Transformercited by 31
  16. 2023LP-OVOD: Open-Vocabulary Object Detection by Linear Probingcited by 29
  17. 2023The devil is in the fine-grained details: Evaluating open-vocabulary object detectors for fine-grained understandingcited by 29
  18. 2024EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensingcited by 29
  19. 2023Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentationcited by 28
  20. 2024OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusioncited by 28
  21. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  22. 2024InstaGen: Enhancing Object Detection by Training on Synthetic Datasetcited by 26
  23. 2024Prompt Learning via Meta-Regularizationcited by 26
  24. 2024Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detectioncited by 26
  25. 2023Taming Self-Training for Open-Vocabulary Object Detectioncited by 25
  26. 2023Defense-Prefix for Preventing Typographic Attacks on CLIPcited by 24
  27. 2024Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detectioncited by 24
  28. 2023Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detectioncited by 22
  29. 2023Domain-Controlled Prompt Learningcited by 22
  30. 2023Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learningcited by 22
  31. 2024Image-Text Co-Decomposition for Text-Supervised Semantic Segmentationcited by 21
  32. 2024LaMI-DETR: Open-Vocabulary Detection with Language Model Instructioncited by 19
  33. 2023Three ways to improve feature alignment for open vocabulary detectioncited by 17
  34. 2024Collaborative Novel Object Discovery and Box-Guided Cross-Modal Alignment for Open-Vocabulary 3D Object Detectioncited by 17
  35. 2024Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillationcited by 17
  36. 2024UniMD: Towards Unifying Moment Retrieval and Temporal Action Detectioncited by 16
  37. 2023ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detectioncited by 15
  38. 2023Simple Image-level Classification Improves Open-vocabulary Object Detectioncited by 15

50 most cited of 133 All 133 (JSON)