Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs
Abstract (arXiv)
Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements - posing a critical barrier to their trustworthy deployment. In this work, we propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition. Our framework is grounded in a normative principle of computational rationality, allowing us to model a VLM's generation as a dynamic cognitive trajectory. We design a suite of information-theoretic probes that project this trajectory onto an interpretable, low-dimensional Cognitive State Space. Our central discovery is a governing principle we term the geometric-information duality: a cognitive trajectory's geometric abnormality within this space is fundamentally equivalent to its high information-theoretic surprisal. Hallucination detection is counts as a geometric anomaly detection problem. Evaluated across diverse settings - from rigorous binary QA (POPE) and comprehensive reasoning (MME) to unconstrained open-ended captioning (MS-COCO) - our framework achieves state-of-the-art performance. Crucially, it operates with high efficiency under weak supervision and remains highly robust even when calibration data is heavily contaminated. This approach enables a causal attribution of failures, mapping observable errors to distinct pathological states: perceptual instability (measured by Perceptual Entropy), logical-causal failure (measured by Inferential Conflict), and decisional ambiguity (measured by Decision Entropy). Ultimately, this opens a path toward building AI systems whose reasoning is transparent, auditable, and diagnosable by design.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 47 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
38 of 47 references.
Closest ancestors · 6
Same direction, newest first
- 2025TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction6 Mar
- 2024Multi-Object Hallucination in Vision-Language Models8 Jul
- 2024Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization24 May
- 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
- 2023Evaluating Object Hallucination in Large Vision-Language Models17 May
Other references · 32
- 2014Microsoft COCO: Common Objects in Contextcited by 17,550
- 2022Survey of Hallucination in Natural Language Generationcited by 2,068
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,637
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 798
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 726
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 708
- 2018Object Hallucination in Image Captioningcited by 663
- 2023The Internal State of an LLM Knows When It's Lyingcited by 652
- 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 321
- 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 255
- 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 236
- 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 229
20 more
- 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 157
- 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
- 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 107
- 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 102
- 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 97
- 2024Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?cited by 85
- 2023Selectively Answering Ambiguous Questionscited by 63
- 2024Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Trainingcited by 62
- 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
- 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
- 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
- 2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
- 2024Estimating the Hallucination Rate of Generative AIcited by 20
- 2025Uncertainty Quantification of Large Language Models through Multi-Dimensional Responsescited by 18
- 2025Vid-SME: Membership Inference Attacks against Large Video Understanding Modelscited by 18
- 2025Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completenesscited by 11
- 2026Sponge Tool Attack: Stealthy Denial-of-Efficiency against Tool-Augmented Agentic Reasoningcited by 11
- 2024Visual Riddles: a Commonsense and World Knowledge Challenge for Large Vision and Language Modelscited by 10
- 2024Data Lineage Inference: Uncovering Privacy Vulnerabilities of Dataset Pruningcited by 5
- 2025Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMscited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
- 2026HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs
- 2025From Detection to Diagnosis: Advancing Hallucination Analysis with Automated Data Synthesis
- 2026Where Does Reasoning Break? Step-Level Hallucination Detection via Hidden-State Transport Geometry
- 2026Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models
- 2025Visual hallucination detection in large vision-language models via evidential conflict
- 2025Revisiting Hallucination Detection with Effective Rank-based Uncertainty
- 2025Graphing the Truth: Structured Visualizations for Automated Hallucination Detection in LLMs