mlsanityData to 14 Oct 2026

RVSD: Retrieval Vision Sparse Decoding for Mitigating Visual Hallucinations in Large Vision-Language Models

arXiv 2609.02731 ↗First posted 2 Sep 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large vision-language models have achieved remarkable success in vision-language tasks. However, they remain prone to Visual Hallucinations (VHs), undermining their reliability in real-world applications. Existing solutions typically require curated datasets, additional training, or multi-round decoding, resulting in considerable computational overhead. In this paper, we propose RVSD (underlineRetrieval underlineVision underlineSparse underlineDecoding), a training-free and plug-and-play decoding framework that, for the first time, unifies token sparsification and Semantic-Space Visual Retrieval (SSVR) within a single decoding pass. Within RVSD, we introduce a semantics-directed token selection strategy that selectively sparsifies redundant tokens while preserving critical visual information. We further propose the SSVR mechanism, which reformulates visual compensation as an on-demand cross-modal retrieval process within a shared semantic space. Extensive experiments demonstrate that RVSD achieves state-of-the-art performance in mitigating VHs while maintaining robust suppression capabilities under long-context generation settings. Our code is available here.footnotehttps://github.com/canjie-liu/RVSD

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 23 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

10 of 23 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Locate-then-Sparsify: Attribution Guided Sparse Strategy for Visual Hallucination Mitigation17 Mar
  2. 2025PruneHal: Reducing Hallucinations in Multi-modal Large Language Models through Adaptive KV Cache Pruning22 Oct
  3. 2025Hallucinatory Image Tokens: A Training-free EAZY Approach on Detecting and Mitigating Object Hallucinations in LVLMs10 Mar
  4. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction2 Feb
  5. 2025VASparse: Towards Efficient Visual Hallucination Mitigation via Visual-Aware Token Sparsification11 Jan
  6. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  7. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov

Other references · 3

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models
  2. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
  3. 2025Revisit What You See: Revealing Visual Semantics in Vision Tokens to Guide LVLM Decoding
  4. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models
  5. 2025Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs
  6. 2026Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidance
  7. 2025Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decoding
  8. 2024Seeing is Believing: Mitigating Hallucination in Large Vision-Language Models via CLIP-Guided Decoding