# SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding

arXiv: https://arxiv.org/abs/2512.04643 · HTML: https://mlsanity.com/paper/2512.04643/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 35 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Builds on (most cited first, 30 of 31)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2311.16922/) (2023)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation](https://mlsanity.com/paper/2311.17911/) (2023)
- [Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning](https://mlsanity.com/paper/2306.14565/) (2023)
- [RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback](https://mlsanity.com/paper/2312.00849/) (2023)
- [TempCompass: Do Video LLMs Really Understand Videos?](https://mlsanity.com/paper/2403.00476/) (2024)
- [Image Captioning with Semantic Attention](https://mlsanity.com/paper/1603.03925/) (2016)
- [Detecting and Preventing Hallucinations in Large Vision Language Models](https://mlsanity.com/paper/2308.06394/) (2023)
- [VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models](https://mlsanity.com/paper/2406.16338/) (2024)
- [VidHalluc: Evaluating Temporal Hallucinations in Multimodal Large Language Models for Video Understanding](https://mlsanity.com/paper/2412.03735/) (2024)
- [EventHallusion: Diagnosing Event Hallucinations in Video LLMs](https://mlsanity.com/paper/2409.16597/) (2024)
- [Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models](https://mlsanity.com/paper/2405.17820/) (2024)
- [Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance](https://mlsanity.com/paper/2402.08680/) (2024)
- [Lost in Time: A New Temporal Benchmark for VideoLLMs](https://mlsanity.com/paper/2410.07752/) (2024)
- [Seeing the Arrow of Time in Large Multimodal Models](https://mlsanity.com/paper/2506.03340/) (2025)
- [Temporal Preference Optimization for Long-Form Video Understanding](https://mlsanity.com/paper/2501.13919/) (2025)
- [MASH-VLM: Mitigating Action-Scene Hallucination in Video-LLMs through Disentangled Spatial-Temporal Representations](https://mlsanity.com/paper/2503.15871/) (2025)
- [PaMi-VDPO: Mitigating Video Hallucinations by Prompt-Aware Multi-Instance Video Preference Learning](https://mlsanity.com/paper/2504.05810/) (2025)
- [Mitigating Hallucination in VideoLLMs via Temporal-Aware Activation Engineering](https://mlsanity.com/paper/2505.12826/) (2025)

## Built on by (most cited first, 10 of 10)

- [Reinforcing Consistency in Video MLLMs with Structured Rewards](https://mlsanity.com/paper/2604.01460/) (2026)
- [Enhancing Video Representations with Spatiotemporal-Semantic Residual to Mitigate Hallucinations in Video Large Multimodal Models](https://mlsanity.com/paper/2601.22574/) (2026)
- [Distorted or Fabricated? A Survey on Hallucination in Video LLMs](https://mlsanity.com/paper/2604.12944/) (2026)
- [Caved or Convinced: Temporal Sampling Gates Claim Deference in Video Large Language Models](https://mlsanity.com/paper/2608.03160/) (2026)
- [From Priors to Perception: Grounding Video-LLMs in Physical Reality](https://mlsanity.com/paper/2605.04515/) (2026)
- [VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models](https://mlsanity.com/paper/2608.08622/) (2026)
- [Test-Time Hallucination Control in Large Vision-Language Models](https://mlsanity.com/paper/2608.11474/) (2026)
- [Agentic Video Understanding: A Survey](https://mlsanity.com/paper/2609.31713/) (2026)
- [Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs](https://mlsanity.com/paper/2610.01595/) (2026)
- [Mitigating Video Hallucinations via Attention-Guided Representation Perturbation](https://mlsanity.com/paper/2610.19728/) (2026)

## Abstract (arXiv)

Video Large Language Models (VideoLLMs) have shown remarkable progress in video understanding. However, these models still struggle to effectively perceive and exploit rich temporal information in videos when responding to user queries. Therefore, they often generate descriptions of events that are temporal inconsistent or causally implausible, causing severe hallucination issues. While most prior studies have focused on spatial hallucinations (e.g. object mismatches), temporal reasoning in video understanding remains relatively underexplored. To address this issue, we propose Self-Diagnostic Contrastive Decoding (SEASON), a training-free method that adaptively enhances temporal and spatial faithfulness for each output token. It achieves this by dynamically diagnosing each token's hallucination tendency and applying adaptive contrastive decoding against its corresponding temporal and spatial negatives. Extensive experiments demonstrate that SEASON outperforms all existing training-free hallucination mitigation approaches on three hallucination examination benchmarks, while further improves VideoLLMs across four general video understanding benchmarks. The code will be released upon acceptance.