mlsanityData to 7 Oct 2026

Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions

arXiv 2410.11701 ↗First posted 15 Oct 2024cs.CL cs.AI cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Hallucinations in multimodal large language models (MLLMs) hinder their practical applications. To address this, we propose a Magnifier Prompt (MagPrompt), a simple yet effective method to tackle hallucinations in MLLMs via extremely simple instructions. MagPrompt is based on the following two key principles, which guide the design of various effective prompts, demonstrating robustness: (1) MLLMs should focus more on the image. (2) When there are conflicts between the image and the model's inner knowledge, MLLMs should prioritize the image. MagPrompt is training-free and can be applied to open-source and closed-source models, such as GPT-4o and Gemini-pro. It performs well across many datasets and its effectiveness is comparable or even better than more complex methods like VCD. Furthermore, our prompt design principles and experimental analyses provide valuable insights into multimodal hallucination.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 8 of its 13 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

8 of 13 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset17 Mar
  2. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  3. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  4. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov

Other references · 4

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  4. 2024MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMscited by 62

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations
  2. 2024Piculet: Specialized Models-Guided Hallucination Decrease for MultiModal Large Language Models
  3. 2024Less is More: Mitigating Multimodal Hallucination from an EOS Decision Perspective
  4. 2024Hallucination of Multimodal Large Language Models: A Survey
  5. 2024Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning
  6. 2024Understanding Multimodal Hallucination with Parameter-Free Representation Alignment
  7. 2025GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs
  8. 2026Mechanisms of Prompt-Induced Hallucination in Vision-Language Models