mlsanityData to 7 Oct 2026

Causal Decoding for Hallucination-Resistant Multimodal Large Language Models

arXiv 2602.21441 ↗First posted 24 Feb 2026cs.LG cs.AI cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) deliver detailed responses on vision-language tasks, yet remain susceptible to object hallucination (introducing objects not present in the image), undermining reliability in practice. Prior efforts often rely on heuristic penalties, post-hoc correction, or generic decoding tweaks, which do not directly intervene in the mechanisms that trigger object hallucination and thus yield limited gains. To address this challenge, we propose a causal decoding framework that applies targeted causal interventions during generation to curb spurious object mentions. By reshaping the decoding dynamics to attenuate spurious dependencies, our approach reduces false object tokens while maintaining descriptive quality. Across captioning and QA benchmarks, our framework substantially lowers object-hallucination rates and achieves state-of-the-art faithfulness without degrading overall output quality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 34 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

29 of 34 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  2. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  3. 2024Skip n: A Simple Method to Reduce Hallucination in Large Vision-Language Models2 Feb
  4. 2024A Survey on Hallucination in Large Vision-Language Models1 Feb
  5. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites4 Dec
  6. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  7. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  8. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data22 Nov
  9. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  10. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 19

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  6. 2021Grounded Language-Image Pre-trainingcited by 1,147
  7. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  8. 2018Object Hallucination in Image Captioningcited by 661
  9. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  10. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  11. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  12. 2022RTMDet: An Empirical Study of Designing Real-Time Object Detectorscited by 192
7 more
  1. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  2. 2016Towards Bayesian Deep Learning: A Framework and Some Existing Methodscited by 57
  3. 2024BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Modelscited by 48
  4. 2020Generative Interventions for Causal Learningcited by 45
  5. 2020Causal Discovery from Incomplete Data: A Deep Learning Approachcited by 26
  6. 2023Self-Interpretable Time Series Prediction with Counterfactual Explanationscited by 24
  7. 2022Causal Transportability for Visual Recognitioncited by 23

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Causal-LLaVA: Causal Disentanglement for Mitigating Hallucination in Multimodal Large Language Models
  2. 2025Beyond Single Models: Mitigating Multimodal Hallucinations via Adaptive Token Ensemble Decoding
  3. 2026Mask What Matters: Mitigating Object Hallucinations in Multimodal Large Language Models with Object-Aligned Visual Contrastive Decoding
  4. 2024Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
  5. 2024Seeing is Believing: Mitigating Hallucination in Large Vision-Language Models via CLIP-Guided Decoding
  6. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation
  7. 2023HallE-Control: Controlling Object Hallucination in Large Multimodal Models
  8. 2026NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors