The Role of Background Information in Reducing Object Hallucination in Vision-Language Models: Insights from Cutoff API Prompting
On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs
Abstract (arXiv)
Vision-Language Models (VLMs) occasionally generate outputs that contradict input images, constraining their reliability in real-world applications. While visual prompting is reported to suppress hallucinations by augmenting prompts with relevant area inside an image, the effectiveness in terms of the area remains uncertain. This study analyzes success and failure cases of Attention-driven visual prompting in object hallucination, revealing that preserving background context is crucial for mitigating object hallucination.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 14 references that are in the map. First posted on arXiv 21 Feb 2025.
Builds on
10 of 14 references.
References · 10
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
- 2023What does CLIP know about a red circle? Visual prompt engineering for VLMscited by 218
- 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
- 2023ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Promptscited by 161
- 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 125
- 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 54
- 2024Attention Prompting on Image for Large Vision-Language Modelscited by 23
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models
- 2026Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMs
- 2026Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
- 2025Two Causes, Not One: Rethinking Omission and Fabrication Hallucinations in MLLMs
- 2024Multi-Object Hallucination in Vision-Language Models
- 2026Mechanisms of Prompt-Induced Hallucination in Vision-Language Models
- 2026Rethinking Visual Neglect: Steering via Context-Preference for MLLM Hallucination Mitigation
- 2026When Prompts Override Vision: Prompt-Induced Hallucinations in LVLMs