mlsanityData to 7 Oct 2026

LV-OSD: Language-Vision-Complementary Open-Set Object Detection

arXiv 2605.28271 ↗First posted 27 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Object detection is an important task in computer vision, which aims to detect the objects of interest. through the given category list or query images. In this work, we propose a new problem of language-visual-complementary open-set object detection (LV-OSD), i.e., using the flexible text-based and/or image-based prompts to specify the desired object categories. This setting is more common and practical in real-world applications. For this purpose, we design a dual-branch detection framework, LVDor, which can simultaneously accept both text and image prompts. Specifically, we first build the Multi-modal Prompts (MPr) containing various text descriptions and image samples for each category. Subsequently, to bridge the semantic gap among the input image, text prompts, and image prompts, we design a Target-guided Prompt Dynamic Weighting (TPDW) module. Guided by the prior information of the target image, this module dynamically produces the text and image prompts that best align with the target semantics, achieving precise alignment and effectively reducing the discrepancy between the two modalities, thereby accommodating the LV-OSD setting. We also propose a simple Prompt Random Masking (PRM) mechanism during training to simulate the arbitrary combination of text and/or image prompts in testing. Extensive experimental results verify our problem formulation's reasonability and our method's effectiveness. Prompts and code will be released publicly.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

36 of 52 references.

References · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  6. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  7. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  8. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  9. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  10. 2021ImageNet-21K Pretraining for the Massescited by 607
  11. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  12. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
24 more
  1. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  2. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  3. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 303
  4. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  5. 2022F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Modelscited by 149
  6. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 145
  7. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  8. 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 114
  9. 2023Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformerscited by 91
  10. 2023Object-Aware Distillation Pyramid for Open-Vocabulary Object Detectioncited by 63
  11. 2023Multi-Modal Classifiers for Open-Vocabulary Object Detectioncited by 53
  12. 2023CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detectioncited by 46
  13. 2022Towards Open-Set Object Detection and Discoverycited by 44
  14. 2025DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perceptioncited by 33
  15. 2023Contrastive Feature Masking Open-Vocabulary Vision Transformercited by 31
  16. 2021Dual-Awareness Attention for Few-Shot Object Detectioncited by 29
  17. 2023LP-OVOD: Open-Vocabulary Object Detection by Linear Probingcited by 29
  18. 2023Taming Self-Training for Open-Vocabulary Object Detectioncited by 25
  19. 2024Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detectioncited by 24
  20. 2023Prompt-Guided Transformers for End-to-End Open-Vocabulary Object Detectioncited by 15
  21. 2023ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detectioncited by 15
  22. 2024Fine-Grained Prototypes Distillation for Few-Shot Object Detectioncited by 10
  23. 2022Integrally Migrating Pre-trained Transformer Encoder-decoders for Visual Object Detectioncited by 9
  24. 2022Towards Generalized Few-Shot Open-Set Object Detectioncited by 8

Built on by

Not cited yet.