mlsanityData to 7 Oct 2026

SAKED: Mitigating Hallucination in Large Vision-Language Models via Stability-Aware Knowledge Enhanced Decoding

arXiv 2602.09825 ↗First posted 10 Feb 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Hallucinations in Large Vision-Language Models (LVLMs) pose significant security and reliability risks in real-world applications. Inspired by the observation that humans are more error-prone when uncertain or hesitant, we investigate how instability in a model 's internal knowledge contributes to LVLM hallucinations. We conduct extensive empirical analyses from three perspectives, namely attention heads, model layers, and decoding tokens, and identify three key hallucination patterns: (i) visual activation drift across attention heads, (ii) pronounced knowledge fluctuations across layers, and (iii) visual focus distraction between neighboring output tokens. Building on these findings, we propose Stability-Aware Knowledge-Enhanced Decoding (SAKED), which introduces a layer-wise Knowledge Stability Score (KSS) to quantify knowledge stability throughout the model. By contrasting the most stability-aware and stability-agnostic layers, SAKED suppresses decoding noise and dynamically leverages the most reliable internal knowledge for faithful token generation. Moreover, SAKED is training-free and can be seamlessly integrated into different architectures. Extensive experiments demonstrate that SAKED achieves state-of-the-art performance for hallucination mitigation on various models, tasks, and benchmarks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 50 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

45 of 50 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025SAVER: Mitigating Hallucinations in Large Vision-Language Models via Style-Aware Visual Early Revision5 Aug
  2. 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding10 Jun
  3. 2025Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs26 May
  4. 2025Image Tokens Matter: Mitigating Hallucination in Discrete Tokenizer-based Large Vision-Language Models via Latent Editing24 May
  5. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering5 Feb
  6. 2025Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key16 Jan
  7. 2024Enhancing Vision-Language Model Reliability with Uncertainty-Guided Dropout Decoding9 Dec
  8. 2024Systematic Reward Gap Optimization for Mitigating VLM Hallucinations26 Nov
  9. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation15 Oct
  10. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination6 Oct
  11. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models4 Oct
  12. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
11 more
  1. 2024Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models24 Jun
  2. 2024Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment28 May
  3. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  4. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  5. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance13 Feb
  6. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites4 Dec
  7. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  8. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  9. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  10. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  11. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 22

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  7. 2023A Survey on Multimodal Large Language Modelscited by 898
  8. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  9. 2018Object Hallucination in Image Captioningcited by 661
  10. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  11. 2023How Language Model Hallucinations Can Snowballcited by 310
  12. 2025Why Language Models Hallucinatecited by 205
10 more
  1. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 127
  2. 2024Harmonizing Visual Text Comprehension and Generationcited by 54
  3. 2025Steer LLM Latents for Hallucination Detectioncited by 49
  4. 2024In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigationcited by 43
  5. 2024Reducing Tool Hallucination via Reliability Alignmentcited by 37
  6. 2024FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedbackcited by 30
  7. 2024MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detectioncited by 27
  8. 2024FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Modelscited by 11
  9. 2024Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Modelscited by 7
  10. 2025Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shiftingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence
  2. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models
  3. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  4. 2026Context-Aware Decoding for Faithful Vision-Language Generation
  5. 2024VaLiD: Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding
  6. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction
  7. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
  8. 2025VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering