mlsanityData to 14 Oct 2026

Mitigating Object Hallucination via Robust Local Perception Search

arXiv 2506.06729 ↗First posted 7 Jun 2025cs.CV cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Recent advancements in Multimodal Large Language Models (MLLMs) have enabled them to effectively integrate vision and language, addressing a variety of downstream tasks. However, despite their significant success, these models still exhibit hallucination phenomena, where the outputs appear plausible but do not align with the content of the images. To mitigate this issue, we introduce Local Perception Search (LPS), a decoding method during inference that is both simple and training-free, yet effectively suppresses hallucinations. This method leverages local visual prior information as a value function to correct the decoding process. Additionally, we observe that the impact of the local visual prior on model performance is more pronounced in scenarios with high levels of image noise. Notably, LPS is a plug-and-play approach that is compatible with various models. Extensive experiments on widely used hallucination benchmarks and noisy data demonstrate that LPS significantly reduces the incidence of hallucinations compared to the baseline, showing exceptional performance, particularly in noisy settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 35 references that are in the map. First posted on arXiv 7 Jun 2025.

Builds on

23 of 35 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Enhancing Vision-Language Model Reliability with Uncertainty-Guided Dropout Decoding9 Dec
  2. 2024Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs15 Nov
  3. 2024CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models4 Jun
  4. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  5. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov

Other references · 18

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2018Object Hallucination in Image Captioningcited by 669
  7. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 181
  8. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 125
  9. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 117
  10. 2024Common 7B Language Models Already Possess Strong Math Capabilitiescited by 109
  11. 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 89
  12. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
6 more
  1. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 60
  2. 2024Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Modelscited by 32
  3. 2024Scaling Inference-Time Search with Vision Value Model for Improved Visual Comprehensioncited by 23
  4. 2024LOVA3: Learning to Visual Question Answering, Asking and Assessmentcited by 21
  5. 2024Inference-Time Language Model Alignment via Integrated Value Guidancecited by 19
  6. 2023CLIPTrans: Transferring Visual Knowledge with Pre-trained Models for Multimodal Machine Translationcited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors
  2. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation
  3. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  4. 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding
  5. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction
  6. 2026SDCD: Structure-Disrupted Contrastive Decoding for Mitigating Hallucinations in Large Vision-Language Models
  7. 2025Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models
  8. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models