mlsanityData to 14 Oct 2026

Assessing Color Vision Test in Large Vision-language Models

arXiv 2507.11153 ↗First posted 15 Jul 2025cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

With the widespread adoption of large vision-language models, the capacity for color vision in these models is crucial. However, the color vision abilities of large visual-language models have not yet been thoroughly explored. To address this gap, we define a color vision testing task for large vision-language models and construct a dataset footnoteAnonymous Github Showing some of the data https://anonymous.4open.science/r/color-vision-test-dataset-3BCD that covers multiple categories of test questions and tasks of varying difficulty levels. Furthermore, we analyze the types of errors made by large vision-language models and propose fine-tuning strategies to enhance their performance in color vision tests.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 27 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

10 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?29 Mar
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond24 Aug

Other references · 8

  1. 2024GPT-4o System Cardcited by 4,588
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  5. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  6. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  7. 2024MVP-Bench: Can Large Vision--Language Models Conduct Multi-level Visual Perception Like Humans?cited by 9
  8. 2024ColorFoil: Investigating Color Blindness in Large Vision and Language Modelscited by 7

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025What Color Is the Text? A Benchmark for Hallucination Induced by Image-Embedded Promptcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025ColorBlindnessEval: Can Vision-Language Models Pass Color Blindness Tests?
  2. 2025Diagnosing Vision Language Models' Perception by Leveraging Human Methods for Color Vision Deficiencies
  3. 2024MegaCOIN: Enhancing Medium-Grained Color Perception for Vision-Language Models
  4. 2025Color Names in Vision-Language Models
  5. 2024POINTS: Improving Your Vision-language Model with Affordable Strategies
  6. 2025ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness
  7. 2024What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases
  8. 2026Spatial Colour Mixing Illusions as a Perception Stress Test for Vision-Language Models