Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in Multimodal LLMs
On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs
Abstract (arXiv)
Hallucinations in LLMs--especially in multimodal settings--undermine reliability. We present a rigorous information-geometric framework, grounded in diffusion dynamics, to quantify hallucinations in MLLMs where model outputs are embedded via spectral decompositions of multimodal graph Laplacians, and their gaps to a truth manifold define a semantic distortion metric. We derive Courant-Fischer bounds on a temperature-dependent hallucination profile and use RKHS eigenmodes to obtain modality-aware, interpretable measures that track evolution over prompts and time. This reframes hallucination as quantifiable and bounded, providing a principled basis for evaluation and mitigation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 60 references that are in the map. First posted on arXiv 26 Aug 2025.
Builds on
22 of 60 references.
References · 22
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
- 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
- 2018Object Hallucination in Image Captioningcited by 661
- 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
- 2023Calibrated Language Models Must Hallucinatecited by 114
10 more
- 2022Multimodal learning with graphscited by 54
- 2025Steer LLM Latents for Hallucination Detectioncited by 49
- 2024On Large Language Models' Hallucination with Regard to Known Factscited by 43
- 2025Hallucination Detection in LLMs Using Spectral Features of Attention Mapscited by 42
- 2024Learning on Multimodal Graphs: A Surveycited by 22
- 2024CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Modelscited by 18
- 2025GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluationcited by 17
- 2025FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMscited by 14
- 2025Semantics at an Angle: When Cosine Similarity Works Until It Doesn'tcited by 13
- 2025ORION Grounded in Context: Retrieval-Based Method for Hallucination Detectioncited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models
- 2025Mathematical Analysis of Hallucination Dynamics in Large Language Models: Uncertainty Quantification, Advanced Decoding, and Principled Mitigation
- 2025Revisiting Hallucination Detection with Effective Rank-based Uncertainty
- 2025HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs
- 2024Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations
- 2025Theoretical Foundations and Mitigation of Hallucination in Large Language Models
- 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection
- 2026HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs