mlsanityData to 14 Oct 2026

Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models

arXiv 2505.20569 ↗First posted 26 May 2025cs.CV cs.AI cs.LGcited by 3

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Despite significant advancements in Large Vision-Language Models, Object Hallucination (OH) remains a persistent challenge. Building upon prior studies on contrastive decoding that address this issue without requiring additional model training, we introduce RVCD (Retrieval Visual Contrastive Decoding), an advanced method to suppress OH. RVCD leverages both negative and positive images at the logit level, explicitly referencing AI-generated images designed to represent a single concept. Our approach demonstrates substantial improvements over existing decoding-based methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 35 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

18 of 35 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination21 Mar
  2. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  3. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  4. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  5. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  6. 2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges6 Nov
  7. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training14 Oct

Other references · 11

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2018YOLOv3: An Incremental Improvementcited by 2,659
  7. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  8. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  9. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 198
  10. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 108
  11. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequencescited by 94

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 20263D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decodingcited by 3
  2. 2026MACD: Model-Aware Contrastive Decoding via Counterfactual Datacited by 2
  3. 2026GEASS: Gated Evidence-Adaptive Selective Caption Trust for Vision-Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026SDCD: Structure-Disrupted Contrastive Decoding for Mitigating Hallucinations in Large Vision-Language Models
  2. 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding
  3. 2024Delve into Visual Contrastive Decoding for Hallucination Mitigation of Large Vision-Language Models
  4. 2025MaskCD: Mitigating LVLM Hallucinations by Image Head Masked Contrastive Decoding
  5. 2026Revealing Multi-View Hallucination in Large Vision-Language Models
  6. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
  7. 2025Efficient Contrastive Decoding with Probabilistic Hallucination Detection - Mitigating Hallucinations in Large Vision Language Models -
  8. 2026CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs