mlsanityData to 7 Oct 2026

Spatial Colour Mixing Illusions as a Perception Stress Test for Vision-Language Models

arXiv 2603.06141 ↗First posted 6 Mar 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Vision-language models (VLMs) achieve strong benchmark results, yet can exhibit systematic perceptual weaknesses: structured, large changes to pixel values can cause confident yet nonsensical predictions, even when the underlying scene remains easily recognizable to humans. We study this gap using Spatial Colour Mixing, a programmatic family of colour distortions that overlays structured patterns (in both RGB and Ostwald colour systems) onto natural images. We introduce a framework of eight spatial colour mixing variants and evaluate nine VLMs across three model families on four datasets. Across models and datasets, accuracy degrades sharply with increasing distortion, and scaling the language model does not reliably mitigate the failure. In a human study with 61 participants on an animal recognition dataset, humans substantially outperform VLMs under the same distortions. Finally, we show that a simple human-inspired preprocessing step recovers a meaningful portion of performance for several distortion types, motivating perception-aware preprocessing and tool-use as practical strategies for improving VLM robustness.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 54 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

25 of 54 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Vision Language Models are Biased29 May
  2. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges4 Jan
  3. 2025IllusionBench+: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models1 Jan
  4. 2024Evaluating Model Perception of Color Illusions in Photorealistic Scenes9 Dec
  5. 2024An Introduction to Vision-Language Modeling27 May
  6. 2023PaLI-3 Vision Language Models: Smaller, Faster, Stronger13 Oct
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  8. 2022A Survey of Vision-Language Pre-Trained Models18 Feb

Other references · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  4. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  7. 2025Gemma 3 Technical Reportcited by 1,686
  8. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  9. 2025DINOv3cited by 1,297
  10. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  11. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  12. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
5 more
  1. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 305
  2. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  3. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Usecited by 84
  4. 2024Illusory VQA: Benchmarking and Enhancing Multimodal Models on Visual Illusionscited by 10
  5. 2024Vision-Language Models Align with Human Neural Representations in Concept Processingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness
  2. 2026Human-AI Perceptual Alignment by Playing Hues and Cues
  3. 2025Color Names in Vision-Language Models
  4. 2025Diagnosing Vision Language Models' Perception by Leveraging Human Methods for Color Vision Deficiencies
  5. 2026Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomalies
  6. 2026Illusion-Aware Visual Preprocessing and Anti-Illusion Prompting for Classic Illusion Understanding in Vision-Language Models
  7. 2026Seeing Red, Thinking Bad: Color Bias in Vision Language Models
  8. 2026When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don't