mlsanityData to 14 Oct 2026

Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory

arXiv 2605.24602 ↗First posted 23 May 2026cs.CV cs.AIcited by 3

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood. In this work, we reveal that hallucinations are strongly associated with a human-like attention distraction phenomenon, where humans under divided focus experience degraded visual clarity and produce inaccurate descriptions, while in models the same mechanism manifests as spatial inconsistency in multi-head attention and temporal fading of attention to image tokens during decoding. We further provide theoretical insights that attention dispersion increases model complexity and degrades classification generalization. Motivated by these findings, we propose an Attention-Focused Approach for Improved Image Perception (AFIP), which corrects attention distraction via cross-head attention enrichment and reinforces visual grounding through dynamic historical attention enhancement. Extensive experiments on multiple benchmarks and models validate the effectiveness of AFIP without additional training. Code is available at: https://github.com/MIKUZ12/AFIP.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 65 references that are in the map. First posted on arXiv 23 May 2026.

Builds on

55 of 65 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs11 Mar
  2. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  3. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models4 Oct
  4. 2024Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation1 Aug
  5. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  6. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  7. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models18 Feb
  8. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  9. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  11. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data22 Nov
  12. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Models29 Aug
2 more
  1. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  2. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  12. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
29 more
  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  3. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  4. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  5. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  6. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  7. 2018Object Hallucination in Image Captioningcited by 669
  8. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  9. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  10. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  11. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  12. 2021Inductive Biases and Variable Creation in Self-Attention Mechanismscited by 144
  13. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 116
  14. 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 114
  15. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  16. 2021Let there be a clock on the beach: Reducing Object Hallucination in Image Captioningcited by 76
  17. 2024IM-RAG: Multi-Round Retrieval-Augmented Generation Through Learning Inner Monologuescited by 40
  18. 2024Large Language Models for Medicine: A Surveycited by 38
  19. 2023Geometric Visual Similarity Learning in 3D Medical Image Self-supervised Pre-trainingcited by 37
  20. 2023Exposing and Mitigating Spurious Correlations for Cross-Modal Retrievalcited by 36
  21. 2021MedGPT: Medical Concept Prediction from Clinical Narrativescited by 24
  22. 2022Robustness Implies Generalization via Data-Dependent Generalization Boundscited by 23
  23. 2024VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenescited by 23
  24. 2025Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inferencecited by 23
  25. 2025Multimodal Language Models See Better When They Look Shallowercited by 23
  26. 2023Debiased Fine-Tuning for Vision-language Models by Prompt Regularizationcited by 21
  27. 2025Medical Large Vision Language Models with Multi-Image Visual Abilitycited by 8
  28. 2026Do All Individual Layers Help? An Empirical Study of Task-Interfering Layers in Vision-Language Modelscited by 2
  29. 2025Towards the Generalization of Multi-view Learning: An Information-theoretical Analysiscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026SP-Mind: An Autonomous Reasoning Agent for Spatial Proteomics Analysiscited by 4
  2. 2026Perceive to Hypothesize, Verify to Ground: An Agentic Reasoning Framework for Open-World Geo-Localizationcited by 0
  3. 2026ROT: Rotating Hidden States towards Contextual Vectors for Hallucination Mitigation in LVLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation
  2. 2024Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs
  3. 2025Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models
  4. 2026See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
  5. 2025Mitigating Hallucinations in Vision-Language Models through Image-Guided Head Suppression
  6. 2026Focus Matters: Phase-Aware Suppression for Hallucination in Vision-Language Models
  7. 2025VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering
  8. 2026Attention to details, logits to truth: visual-aware attention and logits enhancement to mitigate hallucinations in LVLMs