mlsanityData to 7 Oct 2026

SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense

arXiv 2510.16596 ↗First posted 18 Oct 2025cs.CV cs.AIcited by 7

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large Vision-Language Models (LVLMs) excel in diverse cross-modal tasks. However, object hallucination, where models produce plausible but inaccurate object descriptions, remains a significant challenge. In contrast to previous work focusing on LLM components, this paper is the first to trace LVLM hallucinations to visual encoders and identifies three key issues: statistical bias, inherent bias, and vulnerability. To address these challenges, we propose SHIELD, a training-free framework that mitigates hallucinations through three strategies: re-weighting visual tokens to reduce statistical bias, introducing noise-derived tokens to counter inherent bias, and applying adversarial attacks with contrastive decoding to address vulnerability. Experiments demonstrate that SHIELD effectively mitigates object hallucinations across diverse benchmarks and LVLM families. Moreover, SHIELD achieves strong performance on the general LVLM benchmark, highlighting its broad applicability. Code is available at https://github.com/hukcc/SHIELD.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 46 of its 53 references that are in the map. First posted on arXiv 18 Oct 2025.

Builds on

46 of 53 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025MCA-LLaVA: Manhattan Causal Attention for Reducing Hallucination in Large Vision-Language Models12 Jul
  2. 2024Mitigating Object Hallucination via Concentric Causal Attention21 Oct
  3. 2024CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMs19 Aug
  4. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models4 Aug
  5. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  6. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  7. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance13 Feb
  8. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  9. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  10. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  11. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  12. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
1 more
  1. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
21 more
  1. 2023Qwen Technical Reportcited by 3,273
  2. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  5. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  6. 2023Vision Transformers Need Registerscited by 844
  7. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  8. 2018Object Hallucination in Image Captioningcited by 661
  9. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  10. 2022UL2: Unifying Language Learning Paradigmscited by 292
  11. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  12. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 127
  13. 2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
  14. 2023ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image using Large Language Modelscited by 116
  15. 2023Embodied Task Planning with Large Language Modelscited by 84
  16. 2021Let there be a clock on the beach: Reducing Object Hallucination in Image Captioningcited by 74
  17. 2024The Neglected Tails in Vision-Language Modelscited by 63
  18. 2023Exposing and Mitigating Spurious Correlations for Cross-Modal Retrievalcited by 35
  19. 2023LLM as A Robotic Brain: Unifying Egocentric Memory and Controlcited by 32
  20. 2023Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPTcited by 28
  21. 2023LLM-Based Human-Robot Collaboration Framework for Manipulation Taskscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidancecited by 5
  2. 2026TGIF: Text-Guided Layer Fusion Mitigates Hallucination in Multimodal LLMscited by 1
  3. 2026Seeing Through Words: Controlling Visual Retrieval Quality with Language Modelscited by 1
  4. 2026Kestrel: Grounding Self-Refinement for LVLM Hallucination Mitigationcited by 1
  5. 2026Structure-Guided Visual Perturbation Neutralization for LVLMscited by 0
  6. 2026Vision-driven Preference Synthesis for Mitigating Hallucinations in VLMscited by 0
  7. 2026PatchGate: Narrowing the Verbalization Gap with Intrinsic Object Inventories in Frozen Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Analyzing and Mitigating Object Hallucination: A Training Bias Perspective
  2. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
  3. 2024Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD)
  4. 2026NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors
  5. 2026HIME: Mitigating Object Hallucinations in LVLMs via Hallucination Insensitivity Model Editing
  6. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination
  7. 2025MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction
  8. 2025Poison as Cure: Visual Noise for Mitigating Object Hallucinations in LVMs