# Open-vocabulary Attribute Detection

arXiv: https://arxiv.org/abs/2211.12914 · HTML: https://mlsanity.com/paper/2211.12914/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 38 of its 68 references that are in the map.
- First posted on arXiv 23 Nov 2022.

## Placed in

- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [RegionCLIP: Region-based Language-Image Pretraining](https://mlsanity.com/paper/2112.09106/) (2021)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Open-Vocabulary Object Detection Using Captions](https://mlsanity.com/paper/2011.10678/) (2020)
- [An Empirical Study of Training End-to-End Vision-and-Language Transformers](https://mlsanity.com/paper/2111.02387/) (2021)
- [Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts](https://mlsanity.com/paper/2111.08276/) (2021)
- [Zero-Shot Learning -- The Good, the Bad and the Ugly](https://mlsanity.com/paper/1703.04394/) (2017)
- [Zero-Shot Object Detection](https://mlsanity.com/paper/1804.04340/) (2018)
- [Attribute Prototype Network for Zero-Shot Learning](https://mlsanity.com/paper/2008.08290/) (2020)
- [Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection](https://mlsanity.com/paper/2207.03482/) (2022)
- [Automatic Spatially-aware Fashion Concept Discovery](https://mlsanity.com/paper/1708.01311/) (2017)
- [Learning From Noisy Large-Scale Datasets With Minimal Supervision](https://mlsanity.com/paper/1701.01619/) (2017)

## Built on by (most cited first, 29 of 29)

- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023)
- [Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models](https://mlsanity.com/paper/2404.07983/) (2024)
- [OvarNet: Towards Open-vocabulary Object Attribute Recognition](https://mlsanity.com/paper/2301.09506/) (2023)
- [CiteTracker: Correlating Image and Text for Visual Tracking](https://mlsanity.com/paper/2308.11322/) (2023)
- [AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregation](https://mlsanity.com/paper/2309.00096/) (2023)
- [The devil is in the fine-grained details: Evaluating open-vocabulary object detectors for fine-grained understanding](https://mlsanity.com/paper/2311.17518/) (2023)
- [Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy](https://mlsanity.com/paper/2402.07270/) (2024)
- [Explaining and Mitigating the Modality Gap in Contrastive Multimodal Learning](https://mlsanity.com/paper/2412.07909/) (2024)
- [Joint Vision-Language Social Bias Removal for CLIP](https://mlsanity.com/paper/2411.12785/) (2024)
- [ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation](https://mlsanity.com/paper/2311.13258/) (2023)
- [Is CLIP the main roadblock for fine-grained open-world perception?](https://mlsanity.com/paper/2404.03539/) (2024)
- [Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environments](https://mlsanity.com/paper/2410.18195/) (2024)
- [Visual Prompting with Iterative Refinement for Design Critique Generation](https://mlsanity.com/paper/2412.16829/) (2024)
- [DynRefer: Delving into Region-level Multimodal Tasks via Dynamic Resolution](https://mlsanity.com/paper/2405.16071/) (2024)
- [Detailed Object Description with Controllable Dimensions](https://mlsanity.com/paper/2411.19106/) (2024)
- [Compositional Caching for Training-free Open-vocabulary Attribute Detection](https://mlsanity.com/paper/2503.19145/) (2025)
- [ELSA: Evaluating Localization of Social Activities in Urban Streets using Open-Vocabulary Detection](https://mlsanity.com/paper/2406.01551/) (2024)
- [Fine-Grained Open-Vocabulary Object Detection with Fined-Grained Prompts: Task, Dataset and Benchmark](https://mlsanity.com/paper/2503.14862/) (2025)
- [LOWA: Localize Objects in the Wild with Attributes](https://mlsanity.com/paper/2305.20047/) (2023)
- [MAC: A Benchmark for Multiple Attributes Compositional Zero-Shot Learning](https://mlsanity.com/paper/2406.12757/) (2024)
- [Evaluating Attribute Comprehension in Large Vision-Language Models](https://mlsanity.com/paper/2408.13898/) (2024)
- [Beyond Attribution: Unified Concept-Level Explanations](https://mlsanity.com/paper/2410.12439/) (2024)
- [Investigating the Role of Attribute Context in Vision-Language Models for Object Recognition and Detection](https://mlsanity.com/paper/2303.10093/) (2023)
- [SPOTS: Stable Placement of Objects with Reasoning in Semi-Autonomous Teleoperation Systems](https://mlsanity.com/paper/2309.13937/) (2023)
- [An Attribute-Enriched Dataset and Auto-Annotated Pipeline for Open Detection](https://mlsanity.com/paper/2409.06300/) (2024)
- [When Visual Signals Mislead: A Mechanistic Study of Attribute Hallucination in Vision-Language Models](https://mlsanity.com/paper/2608.11024/) (2026)
- [FRAME: Factored Retrieval via Attribute Readouts for Object-Centric Scene Memory](https://mlsanity.com/paper/2609.08886/) (2026)
- [Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments](https://mlsanity.com/paper/2609.27076/) (2026)
- [NegT2IBench: When Negation Changes the Picture. A Polarity Benchmark for Text-to-Image Models](https://mlsanity.com/paper/2610.03084/) (2026)

## Abstract (arXiv)

Vision-language modeling has enabled open-vocabulary tasks where predictions can be queried using any text prompt in a zero-shot manner. Existing open-vocabulary tasks focus on object classes, whereas research on object attributes is limited due to the lack of a reliable attribute-focused evaluation benchmark. This paper introduces the Open-Vocabulary Attribute Detection (OVAD) task and the corresponding OVAD benchmark. The objective of the novel task and benchmark is to probe object-level attribute information learned by vision-language models. To this end, we created a clean and densely annotated test set covering 117 attribute classes on the 80 object classes of MS COCO. It includes positive and negative annotations, which enables open-vocabulary evaluation. Overall, the benchmark consists of 1.4 million annotations. For reference, we provide a first baseline method for open-vocabulary attribute detection. Moreover, we demonstrate the benchmark's value by studying the attribute detection performance of several foundation models. Project page https://ovad-benchmark.github.io