mlsanityData to 13 Oct 2026

Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models

arXiv 2603.15557 ↗First posted 16 Mar 2026cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements - posing a critical barrier to their trustworthy deployment. In this work, we propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition. Our framework is grounded in a normative principle of computational rationality, allowing us to model a VLM's generation as a dynamic cognitive trajectory. We design a suite of information-theoretic probes that project this trajectory onto an interpretable, low-dimensional Cognitive State Space. Our central discovery is a governing principle we term the geometric-information duality: a cognitive trajectory's geometric abnormality within this space is fundamentally equivalent to its high information-theoretic surprisal. Hallucination detection is counts as a geometric anomaly detection problem. Evaluated across diverse settings - from rigorous binary QA (POPE) and comprehensive reasoning (MME) to unconstrained open-ended captioning (MS-COCO) - our framework achieves state-of-the-art performance. Crucially, it operates with high efficiency under weak supervision and remains highly robust even when calibration data is heavily contaminated. This approach enables a causal attribution of failures, mapping observable errors to distinct pathological states: perceptual instability (measured by Perceptual Entropy), logical-causal failure (measured by Inferential Conflict), and decisional ambiguity (measured by Decision Entropy). Ultimately, this opens a path toward building AI systems whose reasoning is transparent, auditable, and diagnosable by design.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 47 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

38 of 47 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction6 Mar
  2. 2024Multi-Object Hallucination in Vision-Language Models8 Jul
  3. 2024Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization24 May
  4. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  5. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
  6. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 32

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,068
  3. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,637
  4. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 798
  5. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 726
  6. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 708
  7. 2018Object Hallucination in Image Captioningcited by 663
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 652
  9. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 321
  10. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 255
  11. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 236
  12. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 229
20 more
  1. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 157
  2. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  3. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 107
  4. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 102
  5. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 97
  6. 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
  7. 2023Selectively Answering Ambiguous Questionscited by 63
  8. 2024Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Trainingcited by 62
  9. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  10. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  11. 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
  12. 2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
  13. 2024Estimating the Hallucination Rate of Generative AIcited by 20
  14. 2025Uncertainty Quantification of Large Language Models through Multi-Dimensional Responsescited by 18
  15. 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 18
  16. 2025Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completenesscited by 11
  17. 2026Sponge Tool Attack: Stealthy Denial-of-Efficiency against Tool-Augmented Agentic Reasoningcited by 11
  18. 2024Visual Riddles: a Commonsense and World Knowledge Challenge for Large Vision and Language Modelscited by 10
  19. 2024Data Lineage Inference: Uncovering Privacy Vulnerabilities of Dataset Pruningcited by 5
  20. 2025Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMscited by 4

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
  2. 2026HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs
  3. 2025From Detection to Diagnosis: Advancing Hallucination Analysis with Automated Data Synthesis
  4. 2026Where Does Reasoning Break? Step-Level Hallucination Detection via Hidden-State Transport Geometry
  5. 2026Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
  6. 2025Visual hallucination detection in large vision-language models via evidential conflict
  7. 2025Revisiting Hallucination Detection with Effective Rank-based Uncertainty
  8. 2025Graphing the Truth: Structured Visualizations for Automated Hallucination Detection in LLMs