mlsanityData to 14 Oct 2026

SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinking

arXiv 2506.02803 ↗First posted 3 Jun 2025cs.CL cs.CVcited by 6

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Vision-language models (VLMs) excel in semantic tasks but falter at a core human capability: detecting hidden content in optical illusions or AI-generated images through perceptual adjustments like zooming. We introduce HC-Bench, a benchmark of 112 images with hidden text, objects, and illusions, revealing that leading VLMs achieve near-zero accuracy (0-5.36%)-even with explicit prompting. Humans resolve such ambiguities instinctively, yet VLMs fail due to an overreliance on high-level semantics. Strikingly, we propose SemVink (Semantic Visual Thinking) by simply scaling images to low resolutions (32-128 pixels), which unlocks >99% accuracy by eliminating redundant visual noise. This exposes a critical architectural flaw: VLMs prioritize abstract reasoning over low-level visual operations crucial for real-world robustness. Our work urges a shift toward hybrid models integrating multi-scale processing, bridging the gap between computational vision and human cognition for applications in medical imaging, security, and beyond.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 45 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

25 of 45 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2019Towards VQA Models That Can Readcited by 1,718
  7. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  8. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  9. 2021Multiscale Vision Transformerscited by 897
  10. 2017Countering Adversarial Images using Input Transformationscited by 520
  11. 2021Probing Image-Language Transformers for Verb Understandingcited by 109
  12. 2019GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answeringcited by 78
13 more
  1. 2024FastVLM: Efficient Vision Encoding for Vision Language Modelscited by 77
  2. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 47
  3. 2024EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Modelscited by 41
  4. 2024ViTamin: Designing Scalable Vision Models in the Vision-Language Eracited by 39
  5. 2024RoRA-VLM: Robust Retrieval-Augmented Vision Language Modelscited by 23
  6. 2025IllusionBench+: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Modelscited by 12
  7. 2025Multimodal Mamba: Decoder-only Multimodal State Space Model via Quadratic to Linear Distillationcited by 12
  8. 2024Illusory VQA: Benchmarking and Enhancing Multimodal Models on Visual Illusionscited by 11
  9. 2024RAVEN: Multitask Retrieval Augmented Vision-Language Learningcited by 10
  10. 2025GuardDoor: Safeguarding Against Malicious Diffusion Editing via Protective Backdoorscited by 5
  11. 2025Visual Watermarking in the Era of Diffusion Models: Advances and Challengescited by 4
  12. 2024Spatial Coherence Loss: All Objects Matter in Salient and Camouflaged Object Detectioncited by 1
  13. 2024Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025VACoT: Rethinking Visual Data Augmentation with VLMscited by 9
  2. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6
  3. 2026Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusionscited by 1
  4. 2026ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillationcited by 0
  5. 2026ControlTrace: Recovering Control Fields for Hidden-Content Recognitioncited by 0
  6. 2026When MLLMs miss what humans read: diagnosing and learning to read squint textcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models
  2. 2026Can LVLMs Uncover the Truth Behind Visual Illusions? An Analysis of Perceptual and Reasoning Capabilities
  3. 2026Seeing the Evidence, Missing the Answer: Tool-Guided Vision-Language Models on Visual Illusions
  4. 2026Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomalies
  5. 2025VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs
  6. 2026Illusion-Aware Visual Preprocessing and Anti-Illusion Prompting for Classic Illusion Understanding in Vision-Language Models
  7. 2025VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs
  8. 2026Beyond Shortcuts: Mitigating Visual Illusions in Frozen VLMs via Qualitative Reasoning