# MIRAGE: The Illusion of Visual Understanding

arXiv: https://arxiv.org/abs/2603.21687 · HTML: https://mlsanity.com/paper/2603.21687/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 51 references that are in the map.
- First posted on arXiv 23 Mar 2026.

## Builds on (most cited first, 30 of 40)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://mlsanity.com/paper/2409.02813/) (2024)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [LoRA+: Efficient Low Rank Adaptation of Large Models](https://mlsanity.com/paper/2402.12354/) (2024)
- [NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark](https://mlsanity.com/paper/2310.18018/) (2023)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [Data Contamination: From Memorization to Exploitation](https://mlsanity.com/paper/2203.08242/) (2022)

## Built on by (most cited first, 30 of 65)

- [Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery](https://mlsanity.com/paper/2605.08956/) (2026)
- [Mirage Probes: How Vision Models Fake Visual Understanding](https://mlsanity.com/paper/2606.13870/) (2026)
- [Medical thinking with multiple images](https://mlsanity.com/paper/2604.16506/) (2026)
- [Towards Conversational Medical AI with Eyes, Ears and a Voice](https://mlsanity.com/paper/2605.09272/) (2026)
- [Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR](https://mlsanity.com/paper/2605.30912/) (2026)
- [Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging](https://mlsanity.com/paper/2606.14957/) (2026)
- [MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models](https://mlsanity.com/paper/2606.06696/) (2026)
- [PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?](https://mlsanity.com/paper/2607.28318/) (2026)
- [CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation](https://mlsanity.com/paper/2604.26288/) (2026)
- [NLG Evaluation: Past, Present, Future](https://mlsanity.com/paper/2605.23715/) (2026)
- [Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging](https://mlsanity.com/paper/2606.02809/) (2026)
- [Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillation](https://mlsanity.com/paper/2606.06076/) (2026)
- [Diagnosing Visual Ignorance in Vision-Language Models](https://mlsanity.com/paper/2606.06890/) (2026)
- [When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understanding](https://mlsanity.com/paper/2606.08239/) (2026)
- [Thinking with Visual Grounding](https://mlsanity.com/paper/2606.16122/) (2026)
- [Veritas++: Value-aware On-Policy Distillation for Perception-Enhanced AIGI Detection](https://mlsanity.com/paper/2607.27113/) (2026)
- [Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understanding](https://mlsanity.com/paper/2607.28516/) (2026)
- [Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models](https://mlsanity.com/paper/2604.03179/) (2026)
- [Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time](https://mlsanity.com/paper/2605.01766/) (2026)
- [Do multimodal models imagine electric sheep?](https://mlsanity.com/paper/2605.09693/) (2026)
- [Medical Context Distorts Decisions in Clinical Vision Language Models](https://mlsanity.com/paper/2605.17436/) (2026)
- [Detect Before You Leap: Mirage Detection in Vision-Language Models](https://mlsanity.com/paper/2606.00435/) (2026)
- [From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQA](https://mlsanity.com/paper/2606.30220/) (2026)
- [Which Decisions Low-Bit Quantization Breaks, and How to Predict Them](https://mlsanity.com/paper/2608.06564/) (2026)
- [Evaluating Multimodal Narrative Understanding of Popular Hollywood Films](https://mlsanity.com/paper/2608.21430/) (2026)
- [Knowing Isn't Always Saying: When Do Spatial Encodings Reach Answers in Vision-Language Models?](https://mlsanity.com/paper/2608.22916/) (2026)
- [Is There Knowledge Left to Extract? Evidence of Fragility in Medically Fine-Tuned Vision-Language Models](https://mlsanity.com/paper/2604.09841/) (2026)
- [The Cost of Language: Centroid Erasure Exposes and Exploits Modal Competition in Multimodal Language Models](https://mlsanity.com/paper/2604.14363/) (2026)
- [From Mirage to Grounding: Towards Reliable Multimodal Circuit-to-Verilog Code Generation](https://mlsanity.com/paper/2604.27969/) (2026)
- [MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence](https://mlsanity.com/paper/2605.07919/) (2026)

## Abstract (arXiv)

Multimodal AI systems have achieved remarkable performance across a broad range of real-world tasks, yet the mechanisms underlying visual-language reasoning remain surprisingly poorly understood. We report three findings that challenge prevailing assumptions about how these systems process and integrate visual information. First, Frontier models readily generate detailed image descriptions and elaborate reasoning traces, including pathology-biased clinical findings, for images never provided; we term this phenomenon mirage reasoning. Second, without any image input, models also attain strikingly high scores across general and medical multimodal benchmarks, bringing into question their utility and design. In the most extreme case, our model achieved the top rank on a standard chest X-ray question-answering benchmark without access to any images. Third, when models were explicitly instructed to guess answers without image access, rather than being implicitly prompted to assume images were present, performance declined markedly. Explicit guessing appears to engage a more conservative response regime, in contrast to the mirage regime in which models behave as though images have been provided. These findings expose fundamental vulnerabilities in how visual-language models reason and are evaluated, pointing to an urgent need for private benchmarks that eliminate textual cues enabling non-visual inference, particularly in medical contexts where miscalibrated AI carries the greatest consequence. We introduce B-Clean as a principled solution for fair, vision-grounded evaluation of multimodal AI systems.