mlsanityData to 14 Oct 2026

Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery

arXiv 2609.37263 ↗First posted 29 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

While Large Vision-Language Models (LVLMs) achieve remarkable success, hallucinations remain a significant barrier to their reliable deployment. Recent studies primarily attribute these issues to cross-modal attention imbalances; most solutions therefore focus on reweighting visual tokens or suppressing language priors. However, such approaches often overlook the spectral characteristics of the visual information flow and frequently rely on Contrastive Decoding (CD), which doubles inference time. Instead of following conventional approaches, we identify two distinct hallucination patterns-Perceptual-Semantic Dissociation and Localized Fixation-and propose FLASH (Frequency-Localized Attention SHaping), a training-free and CD-free framework. FLASH utilizes a Spectral Vortex Score to detect vision heads within multi-head attention layers and applies adaptive spectral modulation to rectify the visual information flow during decoding. Empirical results demonstrate that FLASH achieves a superior balance between performance and efficiency compared to SOTA methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 40 of its 48 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

40 of 48 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formats21 Nov
  2. 2025Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens4 Aug
  3. 2025Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding23 May
  4. 2025ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models17 Mar
  5. 2025Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding1 Mar
  6. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models4 Aug
  7. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  8. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  9. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  10. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding28 Feb
  11. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  12. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
5 more
  1. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  2. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  3. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  4. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  5. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 23

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  5. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  6. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  7. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  8. 2018Object Hallucination in Image Captioningcited by 669
  9. 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 419
  10. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  11. 2020Learning in the Frequency Domaincited by 198
  12. 2021Fine-Grained Image Analysis with Deep Learning: A Surveycited by 135
11 more
  1. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 132
  2. 2023Silkie: Preference Distillation for Large Visual Language Modelscited by 130
  3. 2024Efficient Multimodal Learning from Data-centric Perspectivecited by 122
  4. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areascited by 95
  5. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 91
  6. 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 74
  7. 2024RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Modelscited by 60
  8. 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMscited by 55
  9. 2024Debiasing Multimodal Large Language Models via Penalization of Language Priorscited by 37
  10. 2025Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Drivingcited by 10
  11. 2025Medical Large Vision Language Models with Multi-Image Visual Abilitycited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification
  2. 2025Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models
  3. 2025Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation
  4. 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding
  5. 2025Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs
  6. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction
  7. 2025VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering
  8. 2026See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs