Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs
On the map: LLM reasoning → Hallucination detection and mitigation
Abstract (arXiv)
LVLMs have achieved strong multimodal reasoning capabilities but remain prone to hallucinations, producing outputs inconsistent with visual inputs or user instructions. Existing training-free methods, including contrastive decoding and auxiliary expert models, which incur several times more computational overhead and may introduce potential interference, as well as static internal signal enhancement, are often vulnerable to the attention sink phenomenon. We find that internal Positive Attention Dynamics (PAD) in LVLMs naturally reveal semantically core visual regions under the distortions of attention sinks. Based on this, we propose Positive Attention Dynamics Enhancement (PADE), a training-free attention intervention that constructs a PAD map to identify semantically core visual regions, applies per-head Median Absolute Deviation Scaling to adaptively control the intervention strength, and leverages System-Token Compensation to maintain attention to complex user instructions and support long-term output consistency. Experiments on multiple LVLMs and benchmarks show that PADE improves visual grounding and reduces hallucinations, validating the effectiveness of leveraging internal attention dynamics for reliable multimodal reasoning.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 30 of its 56 references that are in the map. First posted on arXiv 17 Feb 2026.
Builds on
30 of 56 references.
Closest ancestors · 1
Same direction, newest first
Other references · 29
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
- 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
- 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
- 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decodingcited by 198
- 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
- 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 171
- 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 139
- 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decodingcited by 118
17 more
- 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 103
- 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
- 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attentioncited by 74
- 2024CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMscited by 49
- 2024Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMscited by 49
- 2025Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decodingcited by 42
- 2025Vision Transformers Don't Need Trained Registerscited by 39
- 2024CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Modelscited by 37
- 2025PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Trainingcited by 37
- 2024STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Question-Answeringcited by 21
- 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decodingcited by 19
- 2025INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Samplingcited by 10
- 2025Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Modelscited by 9
- 2025OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilitiescited by 8
- 2025Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correctioncited by 6
- 2025Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigationcited by 3
- 2026Towards Interpretable Hallucination Analysis and Mitigation in LVLMs via Contrastive Neuron Steeringcited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation
- 2026Focus Matters: Phase-Aware Suppression for Hallucination in Vision-Language Models
- 2026Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy
- 2026See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
- 2026Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models
- 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
- 2025Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs
- 2026SAGE: Sink-Aware Grounded Decoding for Multimodal Hallucination Mitigation