mlsanityData to 14 Oct 2026

Prompt-Guided Transformers for End-to-End Open-Vocabulary Object Detection

arXiv 2303.14386 ↗First posted 25 Mar 2023cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Prompt-OVD is an efficient and effective framework for open-vocabulary object detection that utilizes class embeddings from CLIP as prompts, guiding the Transformer decoder to detect objects in both base and novel classes. Additionally, our novel RoI-based masked attention and RoI pruning techniques help leverage the zero-shot classification ability of the Vision Transformer-based CLIP, resulting in improved detection performance at minimal computational cost. Our experiments on the OV-COCO and OVLVIS datasets demonstrate that Prompt-OVD achieves an impressive 21.2 times faster inference speed than the first end-to-end open-vocabulary detection method (OV-DETR), while also achieving higher APs than four two-stage-based methods operating within similar inference time ranges. Code will be made available soon.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 26 of its 32 references that are in the map. First posted on arXiv 25 Mar 2023.

Builds on

26 of 32 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2017Mask R-CNNcited by 7,277
  9. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  11. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  12. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
14 more
  1. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  2. 2017Focal Loss for Dense Object Detectioncited by 752
  3. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  4. 2020A Survey on Visual Transformercited by 654
  5. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  6. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  7. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  8. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  9. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
  10. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  11. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 146
  12. 2021SOLQ: Segmenting Objects by Learning Queriescited by 68
  13. 2021ViDT: An Efficient and Effective Fully Transformer-based Object Detectorcited by 46
  14. 2022An Extendable, Efficient and Effective Transformer-based Object Detectorcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 2; 2025: 5; 2026: 3.

15 papers.

Sort:

By area:

  1. 2023Towards Open Vocabulary Learning: A Surveycited by 166
  2. 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 118
  3. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 61
  4. 2024OV-DQUO: Open-Vocabulary DETR with Denoising Text Query Training and Open-World Unknown Objects Supervisioncited by 22
  5. 2025Refining CLIP's Spatial Awareness: A Visual-Centric Perspectivecited by 13
  6. 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluationcited by 10
  7. 2023OV-VG: A Benchmark for Open-Vocabulary Visual Groundingcited by 7
  8. 2024SIA-OVD: Shape-Invariant Adapter for Bridging the Image-Region Gap in Open-Vocabulary Detectioncited by 5
  9. 2025C3-OWD: A Curriculum Cross-modal Contrastive Learning Framework for Open-World Detectioncited by 5
  10. 2025ATAS: Any-to-Any Self-Distillation for Enhanced Open-Vocabulary Dense Predictioncited by 2
  11. 2023DST-Det: Simple Dynamic Self-Training for Open-Vocabulary Object Detectioncited by 1
  12. 2025ODOV: Benchmark the Open-Domain Open-Vocabulary Object Detectioncited by 1
3 more
  1. 2026LV-OSD: Language-Vision-Complementary Open-Set Object Detectioncited by 0
  2. 2026SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perceptioncited by 0
  3. 2026Open-Vocabulary Gaze Object Prediction: Benchmark and Methodcited by 0