mlsanityData to 14 Oct 2026

MIRAGE: The Illusion of Visual Understanding

arXiv 2603.21687 ↗First posted 23 Mar 2026cs.AIcited by 65

Not on the map. Why?

Cited by 33 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Multimodal AI systems have achieved remarkable performance across a broad range of real-world tasks, yet the mechanisms underlying visual-language reasoning remain surprisingly poorly understood. We report three findings that challenge prevailing assumptions about how these systems process and integrate visual information. First, Frontier models readily generate detailed image descriptions and elaborate reasoning traces, including pathology-biased clinical findings, for images never provided; we term this phenomenon mirage reasoning. Second, without any image input, models also attain strikingly high scores across general and medical multimodal benchmarks, bringing into question their utility and design. In the most extreme case, our model achieved the top rank on a standard chest X-ray question-answering benchmark without access to any images. Third, when models were explicitly instructed to guess answers without image access, rather than being implicitly prompted to assume images were present, performance declined markedly. Explicit guessing appears to engage a more conservative response regime, in contrast to the mirage regime in which models behave as though images have been provided. These findings expose fundamental vulnerabilities in how visual-language models reason and are evaluated, pointing to an urgent need for private benchmarks that eliminate textual cues enabling non-visual inference, particularly in medical contexts where miscalibrated AI carries the greatest consequence. We introduce B-Clean as a principled solution for fair, vision-grounded evaluation of multimodal AI systems.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 40 of its 51 references that are in the map. First posted on arXiv 23 Mar 2026.

Builds on

40 of 51 references.

References · 40

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  11. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  12. 2020GLU Variants Improve Transformercited by 1,804
28 more
  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  2. 2019Root Mean Square Layer Normalizationcited by 1,480
  3. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  4. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  5. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  6. 2017Mixed Precision Trainingcited by 1,156
  7. 2025OpenAI GPT-5 System Cardcited by 989
  8. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  9. 2023A Survey on Multimodal Large Language Modelscited by 909
  10. 2018Object Hallucination in Image Captioningcited by 669
  11. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  12. 2025MedGemma Technical Reportcited by 436
  13. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  14. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 377
  15. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  16. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 339
  17. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 329
  18. 2022Data Contamination: From Memorization to Exploitationcited by 236
  19. 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
  20. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  21. 2024Vision language models are blind: Failing to translate detailed visual features into wordscited by 147
  22. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 138
  23. 2024Advancing Multimodal Medical Capabilities of Geminicited by 90
  24. 2025Vision Language Models are Biasedcited by 66
  25. 2025MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Researchcited by 32
  26. 2018Shifting the Baseline: Single Modality Performance on Visual Navigation & QAcited by 30
  27. 2025ReXVQA: A Large-scale Visual Question Answering Benchmark for Generalist Chest X-ray Understandingcited by 20
  28. 2024Navigating the landscape of multimodal AI in medicine: a scoping review on technical challenges and clinical applicationscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 65.

65 papers.

Sort:

By area:

  1. 2026Agentic AI Scientists Are Not Built For Autonomous Scientific Discoverycited by 8
  2. 2026Mirage Probes: How Vision Models Fake Visual Understandingcited by 5
  3. 2026Medical thinking with multiple imagescited by 4
  4. 2026Towards Conversational Medical AI with Eyes, Ears and a Voicecited by 4
  5. 2026Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVRcited by 4
  6. 2026Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimagingcited by 4
  7. 2026MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Modelscited by 3
  8. 2026PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?cited by 3
  9. 2026CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretationcited by 2
  10. 2026NLG Evaluation: Past, Present, Futurecited by 2
  11. 2026Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imagingcited by 2
  12. 2026Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillationcited by 2
38 more
  1. 2026Diagnosing Visual Ignorance in Vision-Language Modelscited by 2
  2. 2026When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understandingcited by 2
  3. 2026Thinking with Visual Groundingcited by 2
  4. 2026Veritas++: Value-aware On-Policy Distillation for Perception-Enhanced AIGI Detectioncited by 2
  5. 2026Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understandingcited by 2
  6. 2026Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Modelscited by 1
  7. 2026Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Timecited by 1
  8. 2026Do multimodal models imagine electric sheep?cited by 1
  9. 2026Medical Context Distorts Decisions in Clinical Vision Language Modelscited by 1
  10. 2026Detect Before You Leap: Mirage Detection in Vision-Language Modelscited by 1
  11. 2026From Accuracy to Visual Dependence: Auditing and Filtering Modality Collapse in Traffic VideoQAcited by 1
  12. 2026Which Decisions Low-Bit Quantization Breaks, and How to Predict Themcited by 1
  13. 2026Evaluating Multimodal Narrative Understanding of Popular Hollywood Filmscited by 1
  14. 2026Knowing Isn't Always Saying: When Do Spatial Encodings Reach Answers in Vision-Language Models?cited by 1
  15. 2026Is There Knowledge Left to Extract? Evidence of Fragility in Medically Fine-Tuned Vision-Language Modelscited by 0
  16. 2026The Cost of Language: Centroid Erasure Exposes and Exploits Modal Competition in Multimodal Language Modelscited by 0
  17. 2026From Mirage to Grounding: Towards Reliable Multimodal Circuit-to-Verilog Code Generationcited by 0
  18. 2026MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidencecited by 0
  19. 2026LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answeringcited by 0
  20. 2026The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Spacecited by 0
  21. 2026Do Composed Image Retrieval Benchmarks Require Multimodal Composition?cited by 0
  22. 2026UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretationcited by 0
  23. 2026NeuroQA: A Large-Scale Image-Grounded Benchmark for 3D Brain MRI Understandingcited by 0
  24. 2026VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenescited by 0
  25. 2026CardioLens: Revealing the Clinical Reality Gap of MLLMs via Multi-Sequence Cardiac MRI Evaluationscited by 0
  26. 2026Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Modelscited by 0
  27. 2026Robusto-2: Benchmarking Humans & VLMs for Autonomous Driving in Lima & New York Citycited by 0
  28. 2026PathScale-R1: Cross-scale Reasoning for Pathological Image Analysiscited by 0
  29. 2026Hearsay: Vision-Language Medical Diagnoses Without an Imagecited by 0
  30. 2026Location-Aware Fine-Grained Representation Learning for Medical Vision Foundation Modelscited by 0
  31. 2026Multi-View Relational Distillation for Spatial Reasoning with Vision-Language Modelscited by 0
  32. 2026When Visual Signals Mislead: A Mechanistic Study of Attribute Hallucination in Vision-Language Modelscited by 0
  33. 2026NARU: A Benchmark for NARrative Evolution and Cultural Nuance Understanding in Japanese Extreme Long Videocited by 0
  34. 2026PathoArgus: Advancing Evidence-Grounded Long-Context Visual Reasoning across Gigapixel Whole-Slide and Multi-Slide Case Contextscited by 0
  35. 2026V-REX: Efficient Specialist VLM Training for Veterinary X-Rayscited by 0
  36. 2026Is Visual Prompting All You Need? Studying VLM Spatial Reasoning under Progressive Visual Scaffoldscited by 0
  37. 2026VIPER: An Expert-Curated Benchmark for Vision-Language Models in Veterinary Pathologycited by 0
  38. 2026Framework and Benchmark for Code-Driven Agentic Testing in Web Developmentcited by 0

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens
  2. 2024Beyond the Hype: A dispassionate look at vision-language models in medical scenario
  3. 2026CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models
  4. 2026MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
  5. 2025MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models
  6. 2025SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinking
  7. 2026Position: The Systemic Lack of Agency in Visual Reasoning
  8. 2025Draft and Refine with Visual Experts