mlsanityData to 7 Oct 2026

Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens

arXiv 2411.16724 ↗First posted 23 Nov 2024cs.CVcited by 139

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Hallucinations in Large Vision-Language Models (LVLMs) significantly undermine their reliability, motivating researchers to explore the causes of hallucination. However, most studies primarily focus on the language aspect rather than the visual. In this paper, we address how LVLMs process visual information and whether this process causes hallucination. Firstly, we use the attention lens to identify the stages at which LVLMs handle visual data, discovering that the middle layers are crucial. Moreover, we find that these layers can be further divided into two stages: ''visual information enrichment'' and ''semantic refinement'' which respectively propagate visual data to object tokens and interpret it through text. By analyzing attention patterns during the visual information enrichment stage, we find that real tokens consistently receive higher attention weights than hallucinated ones, serving as a strong indicator of hallucination. Further examination of multi-head attention maps reveals that hallucination tokens often result from heads interacting with inconsistent objects. Based on these insights, we propose a simple inference-time method that adjusts visual attention by integrating information across various heads. Extensive experiments demonstrate that this approach effectively mitigates hallucinations in mainstream LVLMs without additional training costs. Code is available at https://github.com/ZhangqiJiang07/middle_layers_indicating_hallucinations.

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 57 references that are in the map. First posted on arXiv 23 Nov 2024.

Builds on

47 of 57 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations3 Oct
  2. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  3. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  4. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  5. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  6. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models18 Feb
  7. 2023Hallucination Augmented Contrastive Learning for Multimodal Large Language Model12 Dec
  8. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  9. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  10. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data22 Nov
  11. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  12. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
4 more
  1. 2023HallE-Control: Controlling Object Hallucination in Large Multimodal Models3 Oct
  2. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  3. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  4. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun

Other references · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  11. 2023A Survey on Multimodal Large Language Modelscited by 898
  12. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
19 more
  1. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  2. 2018Object Hallucination in Image Captioningcited by 661
  3. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  4. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  5. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
  6. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  7. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  8. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
  9. 2020Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Modelscited by 90
  10. 2024Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Modelscited by 84
  11. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 78
  12. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  13. 2023Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIPcited by 62
  14. 2024Harmonizing Visual Text Comprehension and Generationcited by 54
  15. 2024A Concept-Based Explainability Framework for Large Multimodal Modelscited by 39
  16. 2023An Empirical Study of CLIP for Text-based Person Searchcited by 38
  17. 2023Geometric Visual Similarity Learning in 3D Medical Image Self-supervised Pre-trainingcited by 36
  18. 2024Wings: Learning Multimodal LLMs without Text-only Forgettingcited by 23
  19. 2023Debiased Fine-Tuning for Vision-language Models by Prompt Regularizationcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 46; 2026: 93.

139 papers.

Sort:

By area:

  1. 2025Steer LLM Latents for Hallucination Detectioncited by 49
  2. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Rewardcited by 45
  3. 2025Visual Representation Alignment for Multimodal Large Language Modelscited by 40
  4. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  5. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearningcited by 31
  6. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 29
  7. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  8. 2025Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Modelscited by 21
  9. 2025MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMscited by 14
  10. 2025Understanding Language Prior of LVLMs by Contrasting Chain-of-Embeddingcited by 13
  11. 2025CompoDistill: Attention Distillation for Compositional Reasoning in Multimodal LLMscited by 13
  12. 2025CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Interventioncited by 11
38 more
  1. 2025GLSim: Detecting Object Hallucinations in LVLMs via Global-Local Similaritycited by 11
  2. 2026Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Modelscited by 11
  3. 2026GEditBench v2: A Human-Aligned Benchmark for General Image Editingcited by 11
  4. 2025Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Groundingcited by 9
  5. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
  6. 2025Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lenscited by 8
  7. 2026Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steeringcited by 8
  8. 2026VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneckcited by 8
  9. 2026Hallucination Begins Where Saliency Dropscited by 8
  10. 2025Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision-Language Modelscited by 7
  11. 2026Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigationcited by 7
  12. 2025Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formatscited by 6
  13. 2025Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correctioncited by 6
  14. 2026Interpretable Debiasing of Vision-Language Models for Social Fairnesscited by 6
  15. 2026Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decodingcited by 6
  16. 2026Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoringcited by 6
  17. 2025The Mechanistic Emergence of Symbol Grounding in Language Modelscited by 5
  18. 2025Segmentation as A Plug-and-Play Capability for Frozen Multimodal LLMscited by 5
  19. 2025HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Modelscited by 5
  20. 2025HalDec-Bench: Benchmarking Hallucination Detector in Image Captioningcited by 5
  21. 2026VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluationcited by 5
  22. 2025CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decodingcited by 4
  23. 2025On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Modelscited by 4
  24. 2026AMPS: Adaptive Modality Preference Steering via Functional Entropycited by 4
  25. 2025LLMs Can Compensate for Deficiencies in Visual Representationscited by 3
  26. 2025Revisit What You See: Revealing Visual Semantics in Vision Tokens to Guide LVLM Decodingcited by 3
  27. 2025METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Modelscited by 3
  28. 2025Pruning Long Chain-of-Thought of Large Reasoning Models via Small-Scale Preference Optimizationcited by 3
  29. 2025Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localizationcited by 3
  30. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3
  31. 2025Reallocating Attention Across Layers to Reduce Multimodal Hallucinationcited by 3
  32. 2025Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddingscited by 3
  33. 2025SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucinationcited by 3
  34. 2026KVSmooth: Mitigating Hallucination in Multi-modal Large Language Models through Key-Value Smoothingcited by 3
  35. 2026Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMscited by 3
  36. 2026Focusing Where Vision Matters: Selective Training for Large Vision Language Models via Visual Information Gaincited by 3
  37. 2026Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinationscited by 3
  38. 2026Thinking Diffusion: Penalize and Guide Visual-Grounded Reasoning in Diffusion Multimodal Language Modelscited by 3

50 most cited of 139 All 139 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Multi-Object Hallucination in Vision-Language Models
  2. 2026Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
  3. 2026Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy
  4. 2026Focus Matters: Phase-Aware Suppression for Hallucination in Vision-Language Models
  5. 2025Mitigating Hallucination in Large Vision-Language Models through Aligning Attention Distribution to Information Flow
  6. 2024Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs
  7. 2025A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models
  8. 2025Two Causes, Not One: Rethinking Omission and Fabrication Hallucinations in MLLMs