# Do Vision-Language Models Understand Compound Nouns?

arXiv: https://arxiv.org/abs/2404.00419 · HTML: https://mlsanity.com/paper/2404.00419/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 5 of its 34 references that are in the map.
- First posted on arXiv 30 Mar 2024.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 5 of 5)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [Demystifying CLIP Data](https://mlsanity.com/paper/2309.16671/) (2023)
- [From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?](https://mlsanity.com/paper/2305.10568/) (2023)

## Built on by (most cited first, 4 of 4)

- [From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance](https://mlsanity.com/paper/2505.20229/) (2025)
- [Seeing Syntax: Uncovering Syntactic Learning Limitations in Vision-Language Models](https://mlsanity.com/paper/2412.08111/) (2024)
- [Attention Guided Alignment in Efficient Vision-Language Models](https://mlsanity.com/paper/2511.17793/) (2025)
- [How Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey](https://mlsanity.com/paper/2412.08158/) (2024)

## Abstract (arXiv)

Open-vocabulary vision-language models (VLMs) like CLIP, trained using contrastive loss, have emerged as a promising new paradigm for text-to-image retrieval. However, do VLMs understand compound nouns (CNs) (e.g., lab coat) as well as they understand nouns (e.g., lab)? We curate Compun, a novel benchmark with 400 unique and commonly used CNs, to evaluate the effectiveness of VLMs in interpreting CNs. The Compun benchmark challenges a VLM for text-to-image retrieval where, given a text prompt with a CN, the task is to select the correct image that shows the CN among a pair of distractor images that show the constituent nouns that make up the CN. Next, we perform an in-depth analysis to highlight CLIPs' limited understanding of certain types of CNs. Finally, we present an alternative framework that moves beyond hand-written templates for text prompts widely used by CLIP-like models. We employ a Large Language Model to generate multiple diverse captions that include the CN as an object in the scene described by the caption. Our proposed method improves CN understanding of CLIP by 8.25% on Compun. Code and benchmark are available at: https://github.com/sonalkum/Compun