mlsanityData to 7 Oct 2026

Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering

arXiv 2601.05159 ↗First posted 8 Jan 2026cs.CV cs.AIcited by 8

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Object hallucination critically undermines the reliability of Multimodal Large Language Models, often stemming from a fundamental failure in cognitive introspection, where models blindly trust linguistic priors over specific visual evidence. Existing mitigations remain limited: contrastive decoding approaches operate superficially without rectifying internal semantic misalignments, while current latent steering methods rely on static vectors that lack instance-specific precision. We introduce Vision-Language Introspection (VLI), a training-free inference framework that simulates a metacognitive self-correction process. VLI first performs Attributive Introspection to diagnose hallucination risks via probabilistic conflict detection and localize the causal visual anchors. It then employs Interpretable Bi-Causal Steering to actively modulate the inference process, dynamically isolating visual evidence from background noise while neutralizing blind confidence through adaptive calibration. VLI achieves state-of-the-art performance on advanced models, reducing object hallucination rates by 12.67% on MMHal-Bench and improving accuracy by 5.8% on POPE.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 56 references that are in the map. First posted on arXiv 8 Jan 2026.

Builds on

38 of 56 references.

Closest ancestors · 27

Same direction, newest first

  1. 2025MaskCD: Mitigating LVLM Hallucinations by Image Head Masked Contrastive Decoding3 Oct
  2. 2025Exposing Hallucinations To Suppress Them: VLMs Representation Editing With Generative Anchors26 Sep
  3. 2025CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention3 Jun
  4. 2025Seeing It or Not? Interpretable Vision-aware Latent Steering to Mitigate Object Hallucinations23 May
  5. 2025Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding22 May
  6. 2025Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception29 Apr
  7. 2025Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling17 Apr
  8. 2025PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning8 Apr
  9. 2025The Hidden Life of Tokens: Reducing Hallucination of Large Vision-Language Models via Visual Information Steering5 Feb
  10. 2025Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibration4 Feb
  11. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence18 Dec
  12. 2024Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection18 Dec
15 more
  1. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  2. 2024Mitigating Object Hallucination via Concentric Causal Attention21 Oct
  3. 2024Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding17 Oct
  4. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  5. 2024DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer21 Jul
  6. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention18 Jun
  7. 2024Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization24 May
  8. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback22 Apr
  9. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  10. 2024PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset17 Mar
  11. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  12. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding28 Feb
  13. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance13 Feb
  14. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  15. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov

Other references · 11

  1. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  2. 2024Calibrating Language Models with Adaptive Temperature Scalingcited by 57
  3. 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMscited by 38
  4. 2022HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extractioncited by 32
  5. 2025MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representationscited by 22
  6. 2025CoheMark: A Novel Sentence-Level Watermark for Enhanced Text Qualitycited by 17
  7. 2025PMark: Towards Robust and Distortion-free Semantic-level Watermarking with Channel Constraintscited by 14
  8. 2024FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Modelscited by 11
  9. 2025VLA-Mark: A cross modal watermark for large vision-language alignment modelcited by 11
  10. 2025Do BERT-Like Bidirectional Models Still Perform Better on Text Classification in the Era of LLMs?cited by 8
  11. 2025Unlocking Speech Instruction Data Potential with Query Rewritingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026AMPS: Adaptive Modality Preference Steering via Functional Entropycited by 4
  2. 2026A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Modelcited by 3
  3. 2026PSF-Med: Measuring and Explaining Paraphrase Sensitivity in Medical Vision Language Modelscited by 3
  4. 2026Decoding by Perturbation: Mitigating MLLM Hallucinations via Dynamic Textual Perturbationcited by 3
  5. 2026Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMscited by 3
  6. 2026Beyond Dominant Patches: Spatial Credit Redistribution For Grounded Vision-Language Modelscited by 0
  7. 2026Hallucinations Leave a Grounding Signature:Verifier-Guided Decoding for Selective Object Correctioncited by 0
  8. 2026EviAnchor: Mitigating Hallucinations in Large Vision-Language Models via Regional Visual Evidence Compensationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  2. 2026Rethinking Visual Neglect: Steering via Context-Preference for MLLM Hallucination Mitigation
  3. 2026Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models
  4. 2026Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration
  5. 2026HaloProbe: Bayesian Detection and Mitigation of Object Hallucinations in Vision-Language Models
  6. 2025Mitigating Hallucinations in Large Vision-Language Models with Internal Fact-based Contrastive Decoding
  7. 2026Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models
  8. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding