Learning to Decode Against Compositional Hallucination in Video Multimodal Large Language Models
On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs
Abstract (arXiv)
Current research on video hallucination mitigation primarily focuses on isolated error types, leaving compositional hallucinations, arising from incorrect reasoning over multiple interacting spatial and temporal factors largely underexplored. We introduce OmniVCHall, a benchmark designed to systematically evaluate both isolated and compositional hallucinations in video multimodal large language models (VLLMs). OmniVCHall spans diverse video domains, introduces a novel camera-based hallucination type, and defines a fine-grained taxonomy, together with adversarial answer options (e.g., "All are correct" and "None of the above") to prevent shortcut reasoning. The evaluations of 39 representative VLLMs reveal that even advanced models (e.g., Qwen3-VL and GPT-5) exhibit substantial performance degradation. We propose TriCD, a contrastive decoding framework with a triple-pathway calibration mechanism. An adaptive perturbation controller dynamically selects distracting operations to construct negative video variants, while a saliency-guided enhancement module adaptively reinforces grounded token-wise visual evidences. These components are optimized via reinforcement learning to encourage precise decision-making under compositional hallucination settings. Experimental results show that TriCD consistently improves performance across two representative backbones, achieving an average accuracy improvement of over 10%. The data and code can be find at https://github.com/BMRETURN/OmniVCHall.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 58 references that are in the map. First posted on arXiv 31 Jan 2026.
Builds on
42 of 58 references.
Closest ancestors · 9
Same direction, newest first
- 2025ELV-Halluc: Benchmarking Semantic Aggregation Hallucinations in Long Video Understanding29 Aug
- 2025ReCo: Reminder Composition Mitigates Hallucinations in Vision-Language Models27 Jun
- 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding10 Jun
- 2024VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding4 Dec
- 2024EventHallusion: Diagnosing Event Hallucinations in Video LLMs25 Sep
- 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models24 Jun
- 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
- 2023Evaluating Object Hallucination in Large Vision-Language Models17 May
Other references · 33
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2025DINOv3cited by 1,297
21 more
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
- 2023A Survey of Hallucination in Large Foundation Modelscited by 365
- 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
- 2017Deep Reinforcement Learning: An Overviewcited by 327
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
- 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
- 2023Video Understanding with Large Language Models: A Surveycited by 203
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 174
- 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 98
- 2023VIGC: Visual Instruction Generation and Correctioncited by 80
- 2023VideoCon: Robust Video-Language Alignment via Contrast Captionscited by 29
- 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understandingcited by 29
- 2025MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representationscited by 22
- 2024VidHal: Benchmarking Temporal Hallucinations in Vision LLMscited by 19
- 2025ARGUS: Hallucination and Omission Evaluation in Video-LLMscited by 16
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding
- 2025Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineering
- 2025ResNetVLLM-2: Addressing ResNetVLLM's Multi-Modal Hallucinations
- 2025Exploring Hallucination of Large Multimodal Models in Video Understanding: Benchmark, Analysis and Mitigation
- 2026MultiToP: Learning to Patch Visual Tokens to Mitigate Hallucinations in Video Large Multimodal Models
- 2026OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models
- 2026ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluation
- 2026Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation