mlsanityData to 7 Oct 2026

Revealing Multi-View Hallucination in Large Vision-Language Models

arXiv 2603.23934 ↗First posted 25 Mar 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large vision-language models (LVLMs) are increasingly being applied to multi-view image inputs captured from diverse viewpoints. Despite this growing use, current LVLMs often generate incorrect responses due to visual interference from non-target instances or viewpoints, a phenomenon we term multi-view hallucination (MVH). To systematically analyze this problem, we construct MVH-Bench, a benchmark comprising 4.8k question-answer pairs targeting two types of hallucination: cross-instance and cross-view. Empirical results show that MVH is prevalent across recent LVLMs. To address this issue, we propose Reference Shift Contrastive Decoding (RSCD), a training-free decoding technique that suppresses visual interference by generating negative logits through attention masking. Experiments on MVH-Bench with LLaVA-OneVision and Qwen2.5-VL demonstrate that RSCD improves performance by up to 25.7 and 94.8 points over existing hallucination mitigation methods, highlighting the effectiveness of RSCD.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 62 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

38 of 62 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation14 Jun
  2. 2025Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling17 Apr
  3. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence18 Dec
  4. 2024VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding4 Dec
  5. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens23 Nov
  6. 2024DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination6 Oct
  7. 2024Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models24 Jun
  8. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention18 Jun
  9. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov

Other references · 28

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  8. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  9. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  10. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  11. 2018Object Hallucination in Image Captioningcited by 661
  12. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
16 more
  1. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  2. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  3. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
  4. 2017Embodied Question Answeringcited by 335
  5. 2024An Introduction to Vision-Language Modelingcited by 175
  6. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  7. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
  8. 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 127
  9. 2023Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorldcited by 65
  10. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  11. 2025Depth Any Camera: Zero-Shot Metric Depth Estimation from Any Cameracited by 38
  12. 2024Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videoscited by 32
  13. 2024VidHal: Benchmarking Temporal Hallucinations in Vision LLMscited by 19
  14. 2025O-MaMa: Learning Object Mask Matching between Egocentric and Exocentric Viewscited by 7
  15. 2025Bootstrap Your Own Views: Masked Ego-Exo Modeling for Fine-grained View-invariant Video Representationscited by 6
  16. 2025Towards Comprehensive Scene Understanding: Integrating First and Third-Person Views for LVLMscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
  2. 2024Delve into Visual Contrastive Decoding for Hallucination Mitigation of Large Vision-Language Models
  3. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning
  4. 2025Mixture of Decoding: An Attention-Inspired Adaptive Decoding Strategy to Mitigate Hallucinations in Large Vision-Language Models
  5. 2026CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
  6. 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding
  7. 2025Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models
  8. 2024VaLiD: Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding