mlsanityData to 7 Oct 2026

ChromouVQA: Benchmarking Vision-Language Models under Chromatic Camouflaged Images

arXiv 2512.05137 ↗First posted 30 Nov 2025cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-Language Models (VLMs) have advanced multimodal understanding, yet still struggle when targets are embedded in cluttered backgrounds requiring figure-ground segregation. To address this, we introduce ChromouVQA, a large-scale, multi-task benchmark based on Ishihara-style chromatic camouflaged images. We extend classic dot plates with multiple fill geometries and vary chromatic separation, density, size, occlusion, and rotation, recording full metadata for reproducibility. The benchmark covers nine vision-question-answering tasks, including recognition, counting, comparison, and spatial reasoning. Evaluations of humans and VLMs reveal large gaps, especially under subtle chromatic contrast or disruptive geometric fills. We also propose a model-agnostic contrastive recipe aligning silhouettes with their camouflaged renderings, improving recovery of global shapes. ChromouVQA provides a compact, controlled benchmark for reproducible evaluation and extension. Code and dataset are available at https://github.com/Chromou-VQA-Benchmark/Chromou-VQA.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 42 references that are in the map. First posted on arXiv 30 Nov 2025.

Builds on

31 of 42 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Evaluating Model Perception of Color Illusions in Photorealistic Scenes9 Dec
  2. 2024The Illusion-Illusion: Vision Language Models See Illusions Where There Are None7 Dec
  3. 2024IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models23 Mar
  4. 2023Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?31 Oct

Other references · 27

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  7. 2015VQA: Visual Question Answeringcited by 2,489
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  10. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  11. 2025Qwen2.5-Omni Technical Reportcited by 704
  12. 2024Phi-4 Technical Reportcited by 675
15 more
  1. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  2. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  3. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  4. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  5. 2025CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuningcited by 31
  6. 2023Tackling Data Bias in MUSIC-AVQA: Crafting a Balanced Dataset for Unbiased Question-Answeringcited by 28
  7. 2023MuseChat: A Conversational Music Recommendation System for Videoscited by 21
  8. 2025Time Blindness: Why Video-Language Models Can't See What Humans Can?cited by 17
  9. 2024Detecting Misinformation in Multimedia Content through Cross-Modal Entity Consistency: A Dual Learning Approachcited by 12
  10. 2024Let Community Rules Be Reflected in Online Content Moderationcited by 9
  11. 2024Mamba Fusion: Learning Actions Through Questioningcited by 5
  12. 2024VAGUE: Visual Contexts Clarify Ambiguous Expressionscited by 5
  13. 2025Every Image Listens, Every Image Dances: Music-Driven Image Animationcited by 4
  14. 2022CP-PINNs: Data-Driven Changepoints Detection in PDEs Using Online Optimized Physics-Informed Neural Networkscited by 3
  15. 2025Improving Topic Modeling of Social Media Short Texts with Rephrasing: A Case Study of COVID-19 Related Tweetscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025RGB-Th-Bench: A Dense benchmark for Visual-Thermal Understanding of Vision Language Models
  2. 2026PRISM-VLM: A Multi-Axis Discriminative Benchmark for Compact Vision-Language Models
  3. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision
  4. 2024Q-Bench+: A Benchmark for Multi-modal Foundation Models on Low-level Vision from Single Images to Pairs
  5. 2023ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks
  6. 2024UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
  7. 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs
  8. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges