# Color Names in Vision-Language Models

arXiv: https://arxiv.org/abs/2509.22524 · HTML: https://mlsanity.com/paper/2509.22524/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 14 of 14)

- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://mlsanity.com/paper/2409.17146/) (2024)
- [T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation](https://mlsanity.com/paper/2307.06350/) (2023)
- [MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe](https://mlsanity.com/paper/2509.18154/) (2025)
- [SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluation](https://mlsanity.com/paper/2412.12693/) (2024)
- [ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness](https://mlsanity.com/paper/2504.10514/) (2025)
- [ColorFoil: Investigating Color Blindness in Large Vision and Language Models](https://mlsanity.com/paper/2405.11685/) (2024)
- [Color in Visual-Language Models: CLIP deficiencies](https://mlsanity.com/paper/2502.04470/) (2025)
- [Diagnosing Vision Language Models' Perception by Leveraging Human Methods for Color Vision Deficiencies](https://mlsanity.com/paper/2505.17461/) (2025)
- [ColorBlindnessEval: Can Vision-Language Models Pass Color Blindness Tests?](https://mlsanity.com/paper/2509.19070/) (2025)
- [Hues and Cues: Human vs. CLIP](https://mlsanity.com/paper/2509.02305/) (2025)

## Built on by (most cited first, 2 of 2)

- [White-Balance First, Adjust Later: Cross-Camera Color Constancy via Vision-Language Evaluation](https://mlsanity.com/paper/2605.19613/) (2026)
- [What Color Is the Text? A Benchmark for Hallucination Induced by Image-Embedded Prompt](https://mlsanity.com/paper/2511.13400/) (2025)

## Abstract (arXiv)

Color serves as a fundamental dimension of human visual perception and a primary means of communicating about objects and scenes. As vision-language models (VLMs) become increasingly prevalent, understanding whether they name colors like humans is crucial for effective human-AI interaction. We present the first systematic evaluation of color naming capabilities across VLMs, replicating classic color naming methodologies using 957 color samples across five representative models. Our results show that while VLMs achieve high accuracy on prototypical colors from classical studies, performance drops significantly on expanded, non-prototypical color sets. We identify 21 common color terms that consistently emerge across all models, revealing two distinct approaches: constrained models using predominantly basic terms versus expansive models employing systematic lightness modifiers. Cross-linguistic analysis across nine languages demonstrates severe training imbalances favoring English and Chinese, with hue serving as the primary driver of color naming decisions. Finally, ablation studies reveal that language model architecture significantly influences color naming independent of visual processing capabilities.