# SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinking

arXiv: https://arxiv.org/abs/2506.02803 · HTML: https://mlsanity.com/paper/2506.02803/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Probing Image-Language Transformers for Verb Understanding](https://mlsanity.com/paper/2106.09141/) (2021)
- [GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering](https://mlsanity.com/paper/1902.09506/) (2019)
- [FastVLM: Efficient Vision Encoding for Vision Language Models](https://mlsanity.com/paper/2412.13303/) (2024)
- [A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges](https://mlsanity.com/paper/2501.02189/) (2025)
- [EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models](https://mlsanity.com/paper/2403.10378/) (2024)
- [ViTamin: Designing Scalable Vision Models in the Vision-Language Era](https://mlsanity.com/paper/2404.02132/) (2024)
- [RoRA-VLM: Robust Retrieval-Augmented Vision Language Models](https://mlsanity.com/paper/2410.08876/) (2024)
- [IllusionBench+: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models](https://mlsanity.com/paper/2501.00848/) (2025)
- [Multimodal Mamba: Decoder-only Multimodal State Space Model via Quadratic to Linear Distillation](https://mlsanity.com/paper/2502.13145/) (2025)
- [Illusory VQA: Benchmarking and Enhancing Multimodal Models on Visual Illusions](https://mlsanity.com/paper/2412.08169/) (2024)
- [RAVEN: Multitask Retrieval Augmented Vision-Language Learning](https://mlsanity.com/paper/2406.19150/) (2024)
- [GuardDoor: Safeguarding Against Malicious Diffusion Editing via Protective Backdoors](https://mlsanity.com/paper/2503.03944/) (2025)
- [Visual Watermarking in the Era of Diffusion Models: Advances and Challenges](https://mlsanity.com/paper/2505.08197/) (2025)
- [Spatial Coherence Loss: All Objects Matter in Salient and Camouflaged Object Detection](https://mlsanity.com/paper/2402.18698/) (2024)
- [Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models](https://mlsanity.com/paper/2411.06287/) (2024)

## Built on by (most cited first, 6 of 6)

- [VACoT: Rethinking Visual Data Augmentation with VLMs](https://mlsanity.com/paper/2512.02361/) (2025)
- [Gated Differentiable Working Memory for Long-Context Language Modeling](https://mlsanity.com/paper/2601.12906/) (2026)
- [Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusions](https://mlsanity.com/paper/2607.19061/) (2026)
- [ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation](https://mlsanity.com/paper/2608.20122/) (2026)
- [ControlTrace: Recovering Control Fields for Hidden-Content Recognition](https://mlsanity.com/paper/2609.34807/) (2026)
- [When MLLMs miss what humans read: diagnosing and learning to read squint text](https://mlsanity.com/paper/2610.13465/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs) excel in semantic tasks but falter at a core human capability: detecting hidden content in optical illusions or AI-generated images through perceptual adjustments like zooming. We introduce HC-Bench, a benchmark of 112 images with hidden text, objects, and illusions, revealing that leading VLMs achieve near-zero accuracy (0-5.36%)-even with explicit prompting. Humans resolve such ambiguities instinctively, yet VLMs fail due to an overreliance on high-level semantics. Strikingly, we propose SemVink (Semantic Visual Thinking) by simply scaling images to low resolutions (32-128 pixels), which unlocks >99% accuracy by eliminating redundant visual noise. This exposes a critical architectural flaw: VLMs prioritize abstract reasoning over low-level visual operations crucial for real-world robustness. Our work urges a shift toward hybrid models integrating multi-scale processing, bridging the gap between computational vision and human cognition for applications in medical imaging, security, and beyond.