mlsanityData to 14 Oct 2026

Efficient Contrastive Decoding with Probabilistic Hallucination Detection - Mitigating Hallucinations in Large Vision Language Models -

arXiv 2504.12137 ↗First posted 16 Apr 2025cs.CV cs.AI cs.CLcited by 1

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Despite recent advances in Large Vision Language Models (LVLMs), these models still suffer from generating hallucinatory responses that do not align with the visual input provided. To mitigate such hallucinations, we introduce Efficient Contrastive Decoding (ECD), a simple method that leverages probabilistic hallucination detection to shift the output distribution towards contextually accurate answers at inference time. By contrasting token probabilities and hallucination scores, ECD subtracts hallucinated concepts from the original distribution, effectively suppressing hallucinations. Notably, our proposed method can be applied to any open-source LVLM and does not require additional LVLM training. We evaluate our method on several benchmark datasets and across different LVLMs. Our experiments show that ECD effectively mitigates hallucinations, outperforming state-of-the-art methods with respect to performance on LVLM benchmarks and computation time.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 47 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

39 of 47 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification29 May
  2. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  3. 2024ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models24 Mar
  4. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  5. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models18 Feb
  6. 2023Hallucination Augmented Contrastive Learning for Multimodal Large Language Model12 Dec
  7. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  8. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  9. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models2 Nov
  10. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  11. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  12. 2023Evaluation and Analysis of Hallucination in Large Vision-Language Models29 Aug
3 more
  1. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  2. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  3. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 24

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  8. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  9. 2018Object Hallucination in Image Captioningcited by 669
  10. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  11. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  12. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
12 more
  1. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  2. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  3. 2022Confident Adaptive Language Modelingcited by 310
  4. 2019Depth-Adaptive Transformercited by 236
  5. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  6. 2018Prediction Error Meta Classification in Semantic Segmentation: Detection via Aggregated Dispersion Measures of Softmax Probabilitiescited by 54
  7. 2024A Survey for Foundation Models in Autonomous Drivingcited by 52
  8. 2018Confidence Scoring Using Whitebox Meta-models with Linear Classifier Probescited by 21
  9. 2020MetaDetect: Uncertainty Quantification and Prediction Quality Estimates for Object Detectioncited by 18
  10. 2020Improving Video Instance Segmentation by Light-weight Temporal Uncertainty Estimatescited by 14
  11. 2020YOdar: Uncertainty-based Sensor Fusion for Vehicle Detection with Camera and Radar Sensorscited by 13
  12. 2023Temporal Performance Prediction for Deep Convolutional Long Short-Term Memory Networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
  2. 2024Delve into Visual Contrastive Decoding for Hallucination Mitigation of Large Vision-Language Models
  3. 2026SDCD: Structure-Disrupted Contrastive Decoding for Mitigating Hallucinations in Large Vision-Language Models
  4. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding
  5. 2024CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs
  6. 2024Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD)
  7. 2025MaskCD: Mitigating LVLM Hallucinations by Image Head Masked Contrastive Decoding
  8. 2025The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs?