# MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation

arXiv: https://arxiv.org/abs/2410.11779 · HTML: https://mlsanity.com/paper/2410.11779/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 53 of its 59 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)

## Builds on (most cited first, 30 of 53)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [Aligning Large Multimodal Models with Factually Augmented RLHF](https://mlsanity.com/paper/2309.14525/) (2023)
- [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2311.16922/) (2023)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models](https://mlsanity.com/paper/2403.06764/) (2024)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023)
- [OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation](https://mlsanity.com/paper/2311.17911/) (2023)
- [RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback](https://mlsanity.com/paper/2312.00849/) (2023)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models](https://mlsanity.com/paper/2309.03883/) (2023)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models](https://mlsanity.com/paper/2311.06607/) (2023)
- [A Survey on Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2402.00253/) (2024)
- [Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models](https://mlsanity.com/paper/2402.07865/) (2024)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection](https://mlsanity.com/paper/2402.03744/) (2024)
- [Analyzing and Mitigating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2310.00754/) (2023)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)

## Built on by (most cited first, 30 of 88)

- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)
- [Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainability](https://mlsanity.com/paper/2501.01346/) (2025)
- [Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs](https://mlsanity.com/paper/2505.19678/) (2025)
- [When Thinking Drifts: Evidential Grounding for Robust Video Reasoning](https://mlsanity.com/paper/2510.06077/) (2025)
- [See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Model](https://mlsanity.com/paper/2509.16087/) (2025)
- [RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events](https://mlsanity.com/paper/2509.01907/) (2025)
- [Visual Multi-Agent System: Mitigating Hallucination Snowballing via Visual Flow](https://mlsanity.com/paper/2509.21789/) (2025)
- [Fast Large Language Model Collaborative Decoding via Speculation](https://mlsanity.com/paper/2502.01662/) (2025)
- [When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understanding](https://mlsanity.com/paper/2506.05551/) (2025)
- [Modality Bias in LVLMs: Analyzing and Mitigating Object Hallucination via Attention Lens](https://mlsanity.com/paper/2508.02419/) (2025)
- [Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models](https://mlsanity.com/paper/2602.21704/) (2026)
- [Systematic Reward Gap Optimization for Mitigating VLM Hallucinations](https://mlsanity.com/paper/2411.17265/) (2024)
- [ActionArt: Advancing Multimodal Large Models for Fine-Grained Human-Centric Video Understanding](https://mlsanity.com/paper/2504.18152/) (2025)
- [Direct Visual Grounding by Directing Attention of Visual Tokens](https://mlsanity.com/paper/2511.12738/) (2025)
- [Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correction](https://mlsanity.com/paper/2512.18813/) (2025)
- [Do Images Speak Louder than Words? Investigating the Effect of Textual Misinformation in VLMs](https://mlsanity.com/paper/2601.19202/) (2026)
- [CoFFT: Chain of Foresight-Focus Thought for Visual Language Models](https://mlsanity.com/paper/2509.22010/) (2025)
- [Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs](https://mlsanity.com/paper/2605.02735/) (2026)
- [Enhancing Visual Reliance in Text Generation: A Bayesian Perspective on Mitigating Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2505.19498/) (2025)
- [PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs](https://mlsanity.com/paper/2506.17901/) (2025)
- [On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models](https://mlsanity.com/paper/2510.09008/) (2025)
- [Segmentation as A Plug-and-Play Capability for Frozen Multimodal LLMs](https://mlsanity.com/paper/2510.16785/) (2025)
- [Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigation](https://mlsanity.com/paper/2510.22067/) (2025)
- [SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucination](https://mlsanity.com/paper/2512.07730/) (2025)
- [MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models](https://mlsanity.com/paper/2601.21181/) (2026)
- [HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2506.17587/) (2025)
- [Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localization](https://mlsanity.com/paper/2509.07864/) (2025)
- [Exposing Hallucinations To Suppress Them: VLMs Representation Editing With Generative Anchors](https://mlsanity.com/paper/2509.21997/) (2025)
- [CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decoding](https://mlsanity.com/paper/2509.23379/) (2025)
- [Zero-Shot Fine-Grained Image Classification Using Large Vision-Language Models](https://mlsanity.com/paper/2510.03903/) (2025)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) frequently exhibit hallucination phenomena, but the underlying reasons remain poorly understood. In this paper, we present an empirical analysis and find that, although MLLMs incorrectly generate the objects in the final output, they are actually able to recognize visual objects in the preceding layers. We speculate that this may be due to the strong knowledge priors of the language model suppressing the visual information, leading to hallucinations. Motivated by this, we propose a novel dynamic correction decoding method for MLLMs DeCo, which adaptively selects the appropriate preceding layers and proportionally integrates knowledge into the final layer to adjust the output logits. Note that DeCo is model agnostic and can be seamlessly incorporated with various classic decoding strategies and applied to different MLLMs. We evaluate DeCo on widely-used benchmarks, demonstrating that it can reduce hallucination rates by a large margin compared to baselines, highlighting its potential to mitigate hallucinations. Code is available at https://github.com/zjunlp/DeCo.