mlsanityData to 7 Oct 2026

GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection

arXiv 2603.27014 ↗First posted 27 Mar 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

Abstract (arXiv)

Fine-grained open-vocabulary object detection (FG-OVD) aims to detect novel object categories described by attribute-rich texts. While existing open-vocabulary detectors show promise at the base-category level, they underperform in fine-grained settings due to the semantic entanglement of subjects and attributes in pretrained vision-language model (VLM) embeddings -- leading to over-representation of attributes, mislocalization, and semantic drift in embedding space. We propose GUIDED, a decomposition framework specifically designed to address the semantic entanglement between subjects and attributes in fine-grained prompts. By separating object localization and fine-grained recognition into distinct pathways, HUIDED aligns each subtask with the module best suited for its respective roles. Specifically, given a fine-grained class name, we first use a language model to extract a coarse-grained subject and its descriptive attributes. Then the detector is guided solely by the subject embedding, ensuring stable localization unaffected by irrelevant or overrepresented attributes. To selectively retain helpful attributes, we introduce an attribute embedding fusion module that incorporates attribute information into detection queries in an attention-based manner. This mitigates over-representation while preserving discriminative power. Finally, a region-level attribute discrimination module compares each detected region against full fine-grained class names using a refined vision-language model with a projection head for improved alignment. Extensive experiments on FG-OVD and 3F-OVD benchmarks show that GUIDED achieves new state-of-the-art results, demonstrating the benefits of disentangled modeling and modular optimization. Our code will be released at https://github.com/lijm48/GUIDED.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 43 references that are in the map. First posted on arXiv 27 Mar 2026.

Builds on

36 of 43 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024MarvelOVD: Marrying Object Recognition and Vision-Language Models for Robust Open-Vocabulary Object Detection31 Jul
  2. 2024LaMI-DETR: Open-Vocabulary Detection with Language Model Instruction16 Jul
  3. 2024Retrieval-Augmented Open-Vocabulary Object Detection8 Apr
  4. 2023The devil is in the fine-grained details: Evaluating open-vocabulary object detectors for fine-grained understanding29 Nov
  5. 2023EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignment3 Sep
  6. 2023Aligning Bag of Regions for Open-Vocabulary Object Detection27 Feb
  7. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model28 Mar

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2024GPT-4o System Cardcited by 4,467
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  9. 2021Grounded Language-Image Pre-trainingcited by 1,147
  10. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  11. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
17 more
  1. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  2. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  3. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
  4. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  5. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  6. 2023PACO: Parts and Attributes of Common Objectscited by 146
  7. 2023CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matchingcited by 133
  8. 2023DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignmentcited by 90
  9. 2023Object-Aware Distillation Pyramid for Open-Vocabulary Object Detectioncited by 63
  10. 2024DetCLIPv3: Towards Versatile Generative Open-vocabulary Object Detectioncited by 48
  11. 2024OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusioncited by 28
  12. 2023Taming Self-Training for Open-Vocabulary Object Detectioncited by 25
  13. 2024Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detectioncited by 24
  14. 2024Is CLIP the main roadblock for fine-grained open-world perception?cited by 13
  15. 2024Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-trainingcited by 13
  16. 2024SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detectioncited by 11
  17. 2025Fine-Grained Open-Vocabulary Object Detection with Fined-Grained Prompts: Task, Dataset and Benchmarkcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024HA-FGOVD: Highlighting Fine-grained Attributes via Explicit Linear Composition for Open-Vocabulary Object Detection
  2. 2022Fine-grained Visual-Text Prompt-Driven Self-Training for Open-Vocabulary Object Detection
  3. 2024LLMs Meet VLMs: Boost Open Vocabulary Object Detection with Fine-grained Descriptors
  4. 2024From Open Vocabulary to Open World: Teaching Vision Language Models to Detect Novel Objects
  5. 2025Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models
  6. 2023Investigating the Role of Attribute Context in Vision-Language Models for Object Recognition and Detection
  7. 2025Talk in Pieces, See in Whole: Disentangled and Hierarchical Representation Learning in Language-based Object Detection
  8. 2025Language-driven Fine-grained Retrieval