mlsanityData to 7 Oct 2026

Open-vocabulary Attribute Detection

arXiv 2211.12914 ↗First posted 23 Nov 2022cs.CV cs.LGcited by 28

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

Abstract (arXiv)

Vision-language modeling has enabled open-vocabulary tasks where predictions can be queried using any text prompt in a zero-shot manner. Existing open-vocabulary tasks focus on object classes, whereas research on object attributes is limited due to the lack of a reliable attribute-focused evaluation benchmark. This paper introduces the Open-Vocabulary Attribute Detection (OVAD) task and the corresponding OVAD benchmark. The objective of the novel task and benchmark is to probe object-level attribute information learned by vision-language models. To this end, we created a clean and densely annotated test set covering 117 attribute classes on the 80 object classes of MS COCO. It includes positive and negative annotations, which enables open-vocabulary evaluation. Overall, the benchmark consists of 1.4 million annotations. For reference, we provide a first baseline method for open-vocabulary attribute detection. Moreover, we demonstrate the benchmark's value by studying the attribute detection performance of several foundation models. Project page https://ovad-benchmark.github.io

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 68 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

38 of 68 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  10. 2015VQA: Visual Question Answeringcited by 2,489
  11. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
26 more
  1. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  2. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  3. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  4. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  5. 2016Modeling Context in Referring Expressionscited by 1,054
  6. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  7. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  8. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  9. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 470
  10. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  11. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  12. 2021Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Conceptscited by 255
  13. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 254
  14. 2018Zero-Shot Object Detectioncited by 189
  15. 2020Attribute Prototype Network for Zero-Shot Learningcited by 160
  16. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 145
  17. 2017Automatic Spatially-aware Fashion Concept Discoverycited by 132
  18. 2017Learning From Noisy Large-Scale Datasets With Minimal Supervisioncited by 115
  19. 2022Exploiting Unlabeled Data with Vision and Language Models for Object Detectioncited by 97
  20. 2021Learning to Predict Visual Attributes in the Wildcited by 95
  21. 2020Symmetry and Group in Attribute-Object Compositionscited by 89
  22. 2021Open Vocabulary Object Detection with Pseudo Bounding-Box Labelscited by 71
  23. 2019Locality and compositionality in zero-shot learningcited by 29
  24. 2016Recovering the Missing Link: Predicting Class-Attribute Associations for Unsupervised Zero-Shot Learningcited by 20
  25. 2022Localized Vision-Language Matching for Open-vocabulary Object Detectioncited by 17
  26. 2020Learning to Infer Unseen Single-/Multi-Attribute-Object Compositions with Graph Networkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 14; 2025: 2; 2026: 3.

28 papers.

Sort:

By area:

  1. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  2. 2024Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Modelscited by 55
  3. 2023OvarNet: Towards Open-vocabulary Object Attribute Recognitioncited by 43
  4. 2023CiteTracker: Correlating Image and Text for Visual Trackingcited by 36
  5. 2023AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregationcited by 29
  6. 2023The devil is in the fine-grained details: Evaluating open-vocabulary object detectors for fine-grained understandingcited by 29
  7. 2024Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchycited by 26
  8. 2024Explaining and Mitigating the Modality Gap in Contrastive Multimodal Learningcited by 25
  9. 2024Joint Vision-Language Social Bias Removal for CLIPcited by 17
  10. 2023ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representationcited by 13
  11. 2024Is CLIP the main roadblock for fine-grained open-world perception?cited by 13
  12. 2024Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environmentscited by 11
16 more
  1. 2024Visual Prompting with Iterative Refinement for Design Critique Generationcited by 5
  2. 2024DynRefer: Delving into Region-level Multimodal Tasks via Dynamic Resolutioncited by 3
  3. 2024Detailed Object Description with Controllable Dimensionscited by 3
  4. 2025Compositional Caching for Training-free Open-vocabulary Attribute Detectioncited by 3
  5. 2024ELSA: Evaluating Localization of Social Activities in Urban Streets using Open-Vocabulary Detectioncited by 2
  6. 2025Fine-Grained Open-Vocabulary Object Detection with Fined-Grained Prompts: Task, Dataset and Benchmarkcited by 2
  7. 2023LOWA: Localize Objects in the Wild with Attributescited by 1
  8. 2024MAC: A Benchmark for Multiple Attributes Compositional Zero-Shot Learningcited by 1
  9. 2024Evaluating Attribute Comprehension in Large Vision-Language Modelscited by 1
  10. 2024Beyond Attribution: Unified Concept-Level Explanationscited by 1
  11. 2023Investigating the Role of Attribute Context in Vision-Language Models for Object Recognition and Detectioncited by 0
  12. 2023SPOTS: Stable Placement of Objects with Reasoning in Semi-Autonomous Teleoperation Systemscited by 0
  13. 2024An Attribute-Enriched Dataset and Auto-Annotated Pipeline for Open Detectioncited by 0
  14. 2026When Visual Signals Mislead: A Mechanistic Study of Attribute Hallucination in Vision-Language Modelscited by 0
  15. 2026FRAME: Factored Retrieval via Attribute Readouts for Object-Centric Scene Memorycited by 0
  16. 2026Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection
  2. 2024HA-FGOVD: Highlighting Fine-grained Attributes via Explicit Linear Composition for Open-Vocabulary Object Detection
  3. 2024Transferable and Principled Efficiency for Open-Vocabulary Segmentation
  4. 2026Exploring Hierarchical Consistency and Unbiased Objectness for Open-Vocabulary Object Detection
  5. 2023DST-Det: Simple Dynamic Self-Training for Open-Vocabulary Object Detection
  6. 2023Towards Open Vocabulary Learning: A Survey
  7. 2023Open-Vocabulary Camouflaged Object Segmentation
  8. 2026A Training-Free Guess What Vision Language Model from Snippets to Open-Vocabulary Object Detection