mlsanityData to 7 Oct 2026

Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD)

arXiv 2408.04664 ↗First posted 6 Aug 2024cs.CL cs.AI cs.CVcited by 23

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large Vision-Language Models (LVLMs) are an extension of Large Language Models (LLMs) that facilitate processing both image and text inputs, expanding AI capabilities. However, LVLMs struggle with object hallucinations due to their reliance on text cues and learned object co-occurrence biases. While most research quantifies these hallucinations, mitigation strategies are still lacking. Our study introduces a Language Contrastive Decoding (LCD) algorithm that adjusts LVLM outputs based on LLM distribution confidence levels, effectively reducing object hallucinations. We demonstrate the advantages of LCD in leading LVLMs, showing up to %4 improvement in POPE F1 scores and up to %36 reduction in CHAIR scores on the COCO validation set, while also improving captioning quality scores. Our method effectively improves LVLMs without needing complex post-processing or retraining, and is easily applicable to different models. Our findings highlight the potential of further exploration of LVLM-specific decoding algorithms.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 24 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

11 of 24 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  2. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites4 Dec
  3. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov

Other references · 8

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  7. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 94
  8. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 12; 2026: 10.

23 papers.

Sort:

By area:

  1. 2025Octopus: Alleviating Hallucination via Dynamic Contrastive Decodingcited by 23
  2. 2024Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decodingcited by 21
  3. 2025G2: Guided Generation for Enhanced Output Diversity in LLMscited by 14
  4. 2025Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decodingcited by 6
  5. 2025VIBE: Can a VLM Read the Room?cited by 4
  6. 2025Reallocating Attention Across Layers to Reduce Multimodal Hallucinationcited by 3
  7. 2025Do Vision Encoders Truly Explain Object Hallucination?: Mitigating Object Hallucination via Simple Fine-Grained CLIPScorecited by 2
  8. 2025Mitigating Multimodal Hallucinations via Gradient-based Self-Reflectioncited by 2
  9. 2025Reflective Preference Optimization (RPO): Enhancing On-Policy Alignment via Hint-Guided Reflectioncited by 2
  10. 2026SDCD: Structure-Disrupted Contrastive Decoding for Mitigating Hallucinations in Large Vision-Language Modelscited by 2
  11. 2026Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Modelscited by 2
  12. 2026FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Modelscited by 2
11 more
  1. 2025Toward More Reliable Artificial Intelligence: Reducing Hallucinations in Vision-Language Modelscited by 1
  2. 2025HalCECE: A Framework for Explainable Hallucination Detection through Conceptual Counterfactuals in Image Captioningcited by 0
  3. 2025Rationale-Enhanced Decoding for Multi-modal Chain-of-Thoughtcited by 0
  4. 2025Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflectioncited by 0
  5. 2026Self-Correction Inside the Model: Leveraging Layer Attention to Mitigate Hallucinations in Large Vision Language Modelscited by 0
  6. 2026Segmentation-Based Attention Entropy: Detecting and Mitigating Object Hallucinations in Large Vision-Language Modelscited by 0
  7. 2026Hallucination-aware intermediate representation edit in large vision-language modelscited by 0
  8. 2026HTDC: Hesitation-Triggered Differential Calibration for Mitigating Hallucination in Large Vision-Language Modelscited by 0
  9. 2026The Cost of Language: Centroid Erasure Exposes and Exploits Modal Competition in Multimodal Language Modelscited by 0
  10. 2026Learning from Fine-Grained Visual Discrepancies: Mitigating Multimodal Hallucinations via In-Context Visual Contrastive Optimizationcited by 0
  11. 2026YARD: Y-Architecture Register Decoding for Efficient Hallucination Mitigation in Large Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
  2. 2025MaskCD: Mitigating LVLM Hallucinations by Image Head Masked Contrastive Decoding
  3. 2025Watch Closely: Mitigating Object Hallucinations in Large Vision-Language Models with Disentangled Decoding
  4. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
  5. 2026NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors
  6. 2024A Survey on Hallucination in Large Vision-Language Models
  7. 2025The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs?
  8. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models