mlsanityData to 14 Oct 2026

Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images

arXiv 2506.07184 ↗First posted 8 Jun 2025cs.AI cs.CL cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

While multimodal large language models excel at various tasks, they still suffer from hallucinations, which limit their reliability and scalability for broader domain applications. To address this issue, recent research mainly focuses on objective hallucination. However, for sequential images, besides objective hallucination, there is also behavioral hallucination, which is less studied. This work aims to fill in the gap. We first reveal that behavioral hallucinations mainly arise from two key factors: prior-driven bias and the snowball effect. Based on these observations, we introduce SHE (Sequence Hallucination Eradication), a lightweight, two-stage framework that (1) detects hallucinations via visual-textual alignment check using our proposed adaptive temporal window and (2) mitigates them via orthogonal projection onto the joint embedding space. We also propose a new metric (BEACH) to quantify behavioral hallucination severity. Empirical results on standard benchmarks demonstrate that SHE reduces behavioral hallucination by over 10% on BEACH while maintaining descriptive accuracy.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 45 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

37 of 45 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation18 Nov
  2. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation15 Oct
  3. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models24 Jun
  4. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  5. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding28 Feb
  6. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites4 Dec
  7. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  8. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  9. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  10. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data22 Nov
  11. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision13 Nov
  12. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun

Other references · 25

  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  3. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  4. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  5. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  6. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  7. 2018Object Hallucination in Image Captioningcited by 669
  8. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  9. 2023DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Textcited by 210
  10. 2016Visual Storytellingcited by 176
  11. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 116
  12. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequencescited by 94
13 more
  1. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 46
  2. 2024Locate-then-edit for Multi-hop Factual Recall under Knowledge Editingcited by 37
  3. 2024ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modelingcited by 34
  4. 2025Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducementscited by 32
  5. 2023Visual Writing Prompts: Character-Grounded Story Generation with Curated Image Sequencescited by 31
  6. 2024RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognitioncited by 24
  7. 2023Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Modelscited by 23
  8. 2025CODEMENV: Benchmarking Large Language Models on Code Migrationcited by 13
  9. 2024Understanding Reasoning in Chain-of-Thought from the Hopfieldian Viewcited by 11
  10. 2025Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoningcited by 11
  11. 2024MQA-KEAL: Multi-hop Question Answering under Knowledge Editing for Arabic Languagecited by 9
  12. 2024MONAL: Model Autophagy Analysis for Modeling Human-AI Interactionscited by 7
  13. 2025COMPKE: Complex Question Answering under Knowledge Editingcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Tracing and Mitigating Hallucinations in Multimodal LLMs via Dynamic Attention Localizationcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025MIHBench: Benchmarking and Mitigating Multi-Image Hallucinations in Multimodal Large Language Models
  2. 2025A Survey of Multimodal Hallucination Evaluation and Detection
  3. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering
  4. 2024Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations
  5. 2026Beyond Language Priors: Diagnosing and Fixing Visual-Origin Hallucinations in Multimodal LLM
  6. 2024Understanding Multimodal Hallucination with Parameter-Free Representation Alignment
  7. 2024Unified Hallucination Detection for Multimodal Large Language Models
  8. 2026ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluation