mlsanityData to 14 Oct 2026

Instruction Lens Score: Your Instruction Contributes a Powerful Object Hallucination Detector for Multimodal Large Language Models

arXiv 2605.12258 ↗First posted 12 May 2026cs.LGcited by 1

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Multimodal large language models (MLLMs) have achieved remarkable progress, yet the object hallucination remains a critical challenge for reliable deployment. In this paper, we present an in-depth analysis of instruction token embeddings and reveal that they implicitly encode visual information while effectively filtering erroneous information introduced by misleading visual embeddings. Building on this insight, we propose the Instruction Lens Score (InsLen), which combines a Calibrated Local Score with a Context Consistency Score that measures context consistency of the object tokens. The proposed approach serves as a plug-and-play object hallucination detector without relying on auxiliary models or additional training. Extensive experiments across multiple benchmarks and diverse MLLM architectures demonstrate that InsLen consistently outperforms existing hallucination detection methods, highlighting its effectiveness and robustness. The code is available at https://github.com/Fraserlairh/Instruction-Lens-Score.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 50 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

32 of 50 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models10 Oct
  2. 2025GLSim: Detecting Object Hallucinations in LVLMs via Global-Local Similarity27 Aug
  3. 2025Hallucinatory Image Tokens: A Training-free EAZY Approach on Detecting and Mitigating Object Hallucinations in LVLMs10 Mar
  4. 2025Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding1 Mar
  5. 2024Beyond Logit Lens: Contextual Embeddings for Robust Hallucination Detection & Grounding in VLMs28 Nov
  6. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  7. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination6 Oct
  8. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations3 Oct
  9. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  10. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  11. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  12. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
5 more
  1. 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models2 Nov
  2. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  3. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  4. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  5. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 15

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2024Mixtral of Expertscited by 1,823
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  9. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  10. 2018Object Hallucination in Image Captioningcited by 669
  11. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  12. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 291
3 more
  1. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 244
  2. 2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 131
  3. 2024DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMscited by 75

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Seeing is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual Grounding
  2. 2024Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs
  3. 2024Understanding Multimodal Hallucination with Parameter-Free Representation Alignment
  4. 2025Analyzing and Mitigating Object Hallucination: A Training Bias Perspective
  5. 2022Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training
  6. 2026Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations
  7. 2026HIME: Mitigating Object Hallucinations in LVLMs via Hallucination Insensitivity Model Editing
  8. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites