mlsanityData to 14 Oct 2026

Self-Correction Inside the Model: Leveraging Layer Attention to Mitigate Hallucinations in Large Vision Language Models

arXiv 2603.00437 ↗First posted 28 Feb 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Although Large Vision-Language Models (LVLMs) have made substantial progress, hallucination, where generated text is not grounded in the visual input, remains a challenge. As LVLMs become stronger, previously reported hallucination patterns, such as linguistic bias and overthinking phenomenon, become far less consistent, making the corresponding mitigation techniques substantially less effective. In this paper, we introduce an Internal self-Correction mechanism utilizing Layer Attention (ICLA) that operates directly on hidden states during generation. Each layer selectively retrieves information from all preceding layers through a diagonal cross-layer attention mechanism, enabling self-refinement without any external correction signals. With introducing and training only 0.2M and 0.1M additional parameters on LLaVA1.5-7B and Qwen2.5-VL-7B, ours consistently improves visual grounding across multiple hallucination benchmarks, demonstrating its effectiveness for more advanced LVLMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 48 references that are in the map. First posted on arXiv 28 Feb 2026.

Builds on

35 of 48 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models21 Jun
  2. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models18 May
  3. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models10 Feb
  4. 2024Delve into Visual Contrastive Decoding for Hallucination Mitigation of Large Vision-Language Models9 Dec
  5. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation15 Oct
  6. 2024Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD)6 Aug
  7. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models4 Aug
  8. 2024Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization24 May
  9. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding28 Feb
  10. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  11. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  12. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
1 more
  1. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 22

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  8. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  9. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  10. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  11. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  12. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
10 more
  1. 2023Universal Self-Consistency for Large Language Model Generationcited by 203
  2. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
  3. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
  4. 2024Large Language Models Can Self-Correct with Key Condition Verificationcited by 39
  5. 2024Debiasing Multimodal Large Language Models via Penalization of Language Priorscited by 37
  6. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  7. 2023Cross-Layer Retrospective Retrieving via Layer Attentioncited by 22
  8. 2022Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problemcited by 18
  9. 2024Strengthening Layer Interaction via Dynamic Layer Attentioncited by 10
  10. 2025Adaptive Integrated Layered Attention (AILA)cited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering
  2. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
  3. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
  4. 2025Mitigating Hallucinations in Vision-Language Models through Image-Guided Head Suppression
  5. 2026SAGE: Sink-Aware Grounded Decoding for Multimodal Hallucination Mitigation
  6. 2025Exposing Hallucinations To Suppress Them: VLMs Representation Editing With Generative Anchors
  7. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction
  8. 2026Focus Matters: Phase-Aware Suppression for Hallucination in Vision-Language Models