mlsanityData to 14 Oct 2026

PAS : Prelim Attention Score for Detecting Object Hallucinations in Large Vision--Language Models

arXiv 2511.11502 ↗First posted 14 Nov 2025cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large vision-language models (LVLMs) are powerful, yet they remain unreliable due to object hallucinations. In this work, we show that in many hallucinatory predictions the LVLM effectively ignores the image and instead relies on previously generated output (prelim) tokens to infer new objects. We quantify this behavior via the mutual information between the image and the predicted object conditioned on the prelim, demonstrating that weak image dependence strongly correlates with hallucination. Building on this finding, we introduce the Prelim Attention Score (PAS), a lightweight, training-free signal computed from attention weights over prelim tokens. PAS requires no additional forward passes and can be computed on the fly during inference. Exploiting this previously overlooked signal, PAS achieves state-of-the-art object-hallucination detection across multiple models and datasets, enabling real-time filtering and intervention.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 36 references that are in the map. First posted on arXiv 14 Nov 2025.

Builds on

27 of 36 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025GLSim: Detecting Object Hallucinations in LVLMs via Global-Local Similarity27 Aug
  2. 2025Mitigating Hallucinations in Vision-Language Models through Image-Guided Head Suppression22 May
  3. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  4. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination6 Oct
  5. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations3 Oct
  6. 2024Reference-free Hallucination Detection for Large Vision-Language Models11 Aug
  7. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention18 Jun
  8. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback22 Apr
  9. 2024Unified Hallucination Detection for Multimodal Large Language Models5 Feb
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  11. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  12. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
2 more
  1. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  2. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2020Longformer: The Long-Document Transformercited by 3,413
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  8. 2018Object Hallucination in Image Captioningcited by 669
  9. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 114
  10. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  11. 2024Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Modelscited by 30
  12. 2024Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layerscited by 25
1 more
  1. 2025How Do LLMs Use Their Depth?cited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Test-Time Hallucination Control in Large Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Segmentation-Based Attention Entropy: Detecting and Mitigating Object Hallucinations in Large Vision-Language Models
  2. 2025Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities
  3. 2026HaloProbe: Bayesian Detection and Mitigation of Object Hallucinations in Vision-Language Models
  4. 2026Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations
  5. 2024Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models
  6. 2025Diving into Mitigating Hallucinations from a Vision Perspective for Large Vision-Language Models
  7. 2025Exploring Causes and Mitigation of Hallucinations in Large Vision Language Models
  8. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models