# Mechanisms of Prompt-Induced Hallucination in Vision-Language Models

arXiv: https://arxiv.org/abs/2601.05201 · HTML: https://mlsanity.com/paper/2601.05201/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 40 of its 54 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)

## Builds on (most cited first, 30 of 40)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts](https://mlsanity.com/paper/2305.13300/) (2023)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [Teaching CLIP to Count to Ten](https://mlsanity.com/paper/2302.12066/) (2023)
- [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](https://mlsanity.com/paper/2310.01469/) (2023)
- [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024)
- [On Identifiability in Transformers](https://mlsanity.com/paper/1908.04211/) (2019)
- [The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation](https://mlsanity.com/paper/2312.09085/) (2023)
- [Cognitive Bias in Decision-Making with LLMs](https://mlsanity.com/paper/2403.00811/) (2024)
- [Capturing Failures of Large Language Models via Human Cognitive Biases](https://mlsanity.com/paper/2202.12299/) (2022)
- [Words or Vision: Do Vision-Language Models Have Blind Faith in Text?](https://mlsanity.com/paper/2503.02199/) (2025)
- [Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers](https://mlsanity.com/paper/2109.04448/) (2021)
- [Vision Language Models are Biased](https://mlsanity.com/paper/2505.23941/) (2025)
- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [Measuring and Controlling Instruction (In)Stability in Language Model Dialogs](https://mlsanity.com/paper/2402.10962/) (2024)
- [Personas as a Way to Model Truthfulness in Language Models](https://mlsanity.com/paper/2310.18168/) (2023)
- [Language Models use Lookbacks to Track Beliefs](https://mlsanity.com/paper/2505.14685/) (2025)
- [IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations](https://mlsanity.com/paper/2404.01266/) (2024)
- [Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMs](https://mlsanity.com/paper/2506.09047/) (2025)
- [A Comprehensive Evaluation of Cognitive Biases in LLMs](https://mlsanity.com/paper/2410.15413/) (2024)

## Built on by (most cited first, 5 of 5)

- [Dual-Pathway Circuits of Object Hallucination in Vision-Language Models](https://mlsanity.com/paper/2605.13156/) (2026)
- [Geometric Risk Control for Vision-Language Model OCR](https://mlsanity.com/paper/2603.19790/) (2026)
- [Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception](https://mlsanity.com/paper/2606.06714/) (2026)
- [Prior Directions: Why GUI Grounding Gets Locked in the Past](https://mlsanity.com/paper/2607.26913/) (2026)
- [Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files](https://mlsanity.com/paper/2610.09264/) (2026)

## Abstract (arXiv)

Large vision-language models (VLMs) are highly capable, yet often hallucinate by favoring textual prompts over visual evidence. We study this failure mode in a controlled object-counting setting, where the prompt overstates the number of objects in the image (e.g., asking a model to describe four waterlilies when only three are present). At low object counts, models often correct the overestimation, but as the number of objects increases, they increasingly conform to the prompt regardless of the discrepancy. Through mechanistic analysis of three VLMs, we identify a small set of attention heads whose ablation substantially reduces prompt-induced hallucinations (PIH) by at least 40% without additional training. Across models, PIH-heads mediate prompt copying in model-specific ways. We characterize these differences and show that PIH ablation increases correction toward visual evidence. Our findings offer insights into the internal mechanisms driving prompt-induced hallucinations, revealing model-specific differences in how these behaviors are implemented.