Color Names in Vision-Language Models
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Color serves as a fundamental dimension of human visual perception and a primary means of communicating about objects and scenes. As vision-language models (VLMs) become increasingly prevalent, understanding whether they name colors like humans is crucial for effective human-AI interaction. We present the first systematic evaluation of color naming capabilities across VLMs, replicating classic color naming methodologies using 957 color samples across five representative models. Our results show that while VLMs achieve high accuracy on prototypical colors from classical studies, performance drops significantly on expanded, non-prototypical color sets. We identify 21 common color terms that consistently emerge across all models, revealing two distinct approaches: constrained models using predominantly basic terms versus expansive models employing systematic lightness modifiers. Cross-linguistic analysis across nine languages demonstrates severe training imbalances favoring English and Chinese, with hue serving as the primary driver of color naming decisions. Finally, ablation studies reveal that language model architecture significantly influences color naming independent of visual processing capabilities.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 28 references that are in the map. First posted on arXiv 26 Sep 2025.
Builds on
14 of 28 references.
Closest ancestors · 3
Same direction, newest first
- 2025ColorBlindnessEval: Can Vision-Language Models Pass Color Blindness Tests?23 Sep
- 2025Diagnosing Vision Language Models' Perception by Leveraging Human Methods for Color Vision Deficiencies23 May
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models25 Sep
Other references · 11
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 260
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 186
- 2024SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluationcited by 31
- 2025ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustnesscited by 28
- 2024ColorFoil: Investigating Color Blindness in Large Vision and Language Modelscited by 7
- 2025Color in Visual-Language Models: CLIP deficienciescited by 4
- 2025Hues and Cues: Human vs. CLIPcited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Human-AI Perceptual Alignment by Playing Hues and Cues
- 2024Large Language Models estimate fine-grained human color-concept associations
- 2026Spatial Colour Mixing Illusions as a Perception Stress Test for Vision-Language Models
- 2026When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't
- 2026Seeing Red, Thinking Bad: Color Bias in Vision Language Models
- 2026Modeling Human-Like Color Naming Behavior in Context
- 2024Evaluating Model Perception of Color Illusions in Photorealistic Scenes
- 2026Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs