mlsanityData to 14 Oct 2026

GLSim: Detecting Object Hallucinations in LVLMs via Global-Local Similarity

arXiv 2508.19972 ↗First posted 27 Aug 2025cs.CV cs.AI cs.CLcited by 11

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Object hallucination in large vision-language models presents a significant challenge to their safe deployment in real-world applications. Recent works have proposed object-level hallucination scores to estimate the likelihood of object hallucination; however, these methods typically adopt either a global or local perspective in isolation, which may limit detection reliability. In this paper, we introduce GLSim, a novel training-free object hallucination detection framework that leverages complementary global and local embedding similarity signals between image and text modalities, enabling more accurate and reliable hallucination detection in diverse scenarios. We comprehensively benchmark existing object hallucination detection methods and demonstrate that GLSim achieves superior detection performance, outperforming competitive baselines by a significant margin.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 59 of its 67 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

59 of 67 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024Beyond Logit Lens: Contextual Embeddings for Robust Hallucination Detection & Grounding in VLMs28 Nov
  2. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  3. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation18 Nov
  4. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination6 Oct
  5. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations3 Oct
  6. 2024Reference-free Hallucination Detection for Large Vision-Language Models11 Aug
  7. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  8. 2024Multi-Object Hallucination in Vision-Language Models8 Jul
  9. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  10. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  11. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  12. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models2 Nov
6 more
  1. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
  2. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  3. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Models29 Aug
  4. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  5. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  6. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2024GPT-4o System Cardcited by 4,588
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  10. 2023Qwen Technical Reportcited by 3,344
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
29 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  3. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  5. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  6. 2019Attention is not Explanationcited by 929
  7. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  8. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  9. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  10. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  11. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  12. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  13. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  14. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  15. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  16. 2018Object Hallucination in Image Captioningcited by 669
  17. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  18. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  19. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
  20. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  21. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  22. 2023Language Modeling Is Compressioncited by 225
  23. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 197
  24. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 132
  25. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 99
  26. 2025Steer LLM Latents for Hallucination Detectioncited by 50
  27. 2024ALOHa: A New Measure for Hallucination in Captioning Modelscited by 45
  28. 2024VIAssist: Adapting Multi-modal Large Language Models for Users with Visual Impairmentscited by 25
  29. 2025HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detectioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 10.

11 papers.

  1. 2026LatentLens: Revealing Highly Interpretable Visual Tokens in LLMscited by 15
  2. 2026Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoringcited by 9
  3. 2026VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluationcited by 5
  4. 2026Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attentioncited by 2
  5. 2025PAS : Prelim Attention Score for Detecting Object Hallucinations in Large Vision--Language Modelscited by 1
  6. 2026FaithSCAN: Model-Driven Single-Pass Hallucination Detection for Faithful Visual Question Answeringcited by 1
  7. 2026Instruction Lens Score: Your Instruction Contributes a Powerful Object Hallucination Detector for Multimodal Large Language Modelscited by 1
  8. 2026Hallucinations Leave a Grounding Signature:Verifier-Guided Decoding for Selective Object Correctioncited by 1
  9. 2026TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMscited by 0
  10. 2026VisER: Visual Evidence and Reliance for Object Hallucination Detection in LVLMscited by 0
  11. 2026Detecting Object Hallucinations in Large Vision-Language Models via Cross-Modal Attention Drifts and Mask-Based Verificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations
  2. 2025Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities
  3. 2025Diving into Mitigating Hallucinations from a Vision Perspective for Large Vision-Language Models
  4. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models
  5. 2025HalLoc: Token-level Localization of Hallucinations for Vision Language Models
  6. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training
  7. 2025Hallucinatory Image Tokens: A Training-free EAZY Approach on Detecting and Mitigating Object Hallucinations in LVLMs
  8. 2025Exploring Causes and Mitigation of Hallucinations in Large Vision Language Models