mlsanityData to 7 Oct 2026

Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidance

arXiv 2602.01047 ↗First posted 1 Feb 2026cs.CV cs.AIcited by 5

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large Vision-Language Models (LVLMs) can reason from image-text inputs and perform well in various multimodal tasks. Despite this success, they are affected by language priors and often produce hallucinations. Hallucinations denote generated content that is grammatically and syntactically coherent, yet bears no match or direct relevance to visual input. To address this problem, we propose Residual Decoding (ResDec). It is a novel training-free method that uses historical information to aid decoding. The method relies on the internal implicit reasoning mechanism and token logits evolution mechanism of LVLMs to correct biases. Extensive experiments demonstrate that ResDec effectively suppresses hallucinations induced by language priors, significantly improves visual grounding, and reduces object hallucinations. In addition to mitigating hallucinations, ResDec also performs exceptionally well on comprehensive LVLM benchmarks, highlighting its broad applicability.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 60 of its 82 references that are in the map. First posted on arXiv 1 Feb 2026.

Builds on

60 of 82 references.

Closest ancestors · 27

Same direction, newest first

  1. 2025SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense18 Oct
  2. 2025Mitigating Hallucinations in Multimodal LLMs via Object-aware Preference Optimization27 Aug
  3. 2025Analyzing and Mitigating Object Hallucination: A Training Bias Perspective6 Aug
  4. 2025LISA: A Layer-wise Integration and Suppression Approach for Hallucination Mitigation in Multimodal Large Language Models25 Jul
  5. 2025A Survey of Multimodal Hallucination Evaluation and Detection25 Jul
  6. 2025Mitigating Object Hallucinations via Sentence-Level Early Intervention16 Jul
  7. 2025Robust Multimodal Large Language Models Against Modality Conflict9 Jul
  8. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models1 Jul
  9. 2025Mitigating Hallucinations in Large Vision-Language Models via Entity-Centric Multimodal Preference Optimization4 Jun
  10. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering5 Feb
  11. 2025Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibration4 Feb
  12. 2025Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key16 Jan
15 more
  1. 2024Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization15 Nov
  2. 2024Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding17 Oct
  3. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models4 Oct
  4. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  5. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention18 Jun
  6. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  7. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance13 Feb
  8. 2023Hallucination Augmented Contrastive Learning for Multimodal Large Language Model12 Dec
  9. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  11. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  12. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  13. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  14. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  15. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 33

  1. 2025Qwen2.5-VL Technical Reportcited by 5,418
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  8. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  9. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  10. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  11. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  12. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
21 more
  1. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  2. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  3. 2023MultiModal-GPT: A Vision and Language Model for Dialogue with Humanscited by 255
  4. 2025Deep Think with Confidencecited by 199
  5. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 144
  6. 2019Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editingcited by 108
  7. 2024Calibrated Self-Rewarding Vision Language Modelscited by 77
  8. 2023Revisiting the Role of Language Priors in Vision-Language Modelscited by 50
  9. 2025Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potentialcited by 46
  10. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoningcited by 39
  11. 2025Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Informationcited by 27
  12. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  13. 2024Order Matters: Exploring Order Sensitivity in Multimodal Large Language Modelscited by 19
  14. 2025Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillationcited by 15
  15. 2025Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimizationcited by 14
  16. 2025Modality-Balancing Preference Optimization of Large Multimodal Models by Adversarial Negative Miningcited by 8
  17. 2025GuiLoMo: Allocating Expert Number and Rank for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectorscited by 6
  18. 2024Understanding Multimodal Deep Neural Networks: A Concept Selection Viewcited by 4
  19. 2024Balancing Speciality and Versatility: A Coarse to Fine Framework for Mitigating Catastrophic Forgetting in Large Language Modelscited by 3
  20. 2025From Global to Local: Social Bias Transfer in CLIPcited by 2
  21. 2026MMFormalizer: Multimodal Autoformalization in the Wildcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Knowing Isn't Always Saying: When Do Spatial Encodings Reach Answers in Vision-Language Models?cited by 1
  2. 2026Test-Time Hallucination Control in Large Vision-Language Modelscited by 0
  3. 2026Beyond Information Seeking: Severity-Aware Question Supervision for Proactive Medical Dialoguecited by 0
  4. 2026OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillationcited by 0
  5. 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors
  2. 2025Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models
  3. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models
  4. 2025Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs
  5. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  6. 2025Watch Closely: Mitigating Object Hallucinations in Large Vision-Language Models with Disentangled Decoding
  7. 2025VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering
  8. 2025HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models