mlsanityData to 14 Oct 2026

Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?

arXiv 2609.17248 ↗First posted 15 Sep 2026cs.CVcited by 0

On the map: LLM reasoning → Audio-language model reasoning

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models have demonstrated impressive video understanding, yet their ability to reason over long-form narratives is often masked by visual-centric evaluations and inefficient context processing. Existing benchmarks over-rely on visual heuristics while marginalizing auditory cues, effectively reducing models to "silent observers" that bypass genuine cross-modal reasoning. Moreover, standard dense sampling creates an evidence-context trade-off: increasing frames to capture evidence inevitably leads to attention distraction and token explosion. To bridge these gaps, we present Video-HolmesV2, a novel benchmark designed for Deep Audio-Visual Coupling. Unlike previous works, it enforces an Evidence-Based Evaluation, requiring models to justify answers with precise spatio-temporal audio-visual evidence, thereby reducing confounding effects of guessing and hallucinated evidence. To support this, we introduce: (1) a Multi-Model Cross-Verification pipeline to ensure task rigor; (2) a Spatio-temporal Evidence-Aware Metric for fine-grained calibration. Furthermore, we propose an Audio-Text Guided Token Compression framework. By fusing task intent with auditory anchors, our method distills high-value reasoning cues to mitigate long-context noise. In our evaluation, even strong proprietary models achieve below 60% accuracy, while our approach outperforms comparable open-source omni-models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 44 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

28 of 44 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs12 Oct
  2. 2025Qwen2.5-Omni Technical Report26 Mar

Other references · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  5. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  6. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  7. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  8. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  9. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  10. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  11. 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
  12. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
14 more
  1. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 182
  2. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 173
  3. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
  4. 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 137
  5. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 118
  6. 2024video-SALMONN: Speech-Enhanced Audio-Visual Large Language Modelscited by 102
  7. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  8. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 43
  9. 2025VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoningcited by 42
  10. 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 29
  11. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 25
  12. 2025OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Modelscited by 22
  13. 2025Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenescited by 7
  14. 2025Guiding the Inner Eye: A Framework for Hierarchical and Flexible Visual Grounded Reasoningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos
  2. 2026Semantic and Visual Evidence for Efficient Long-Video Reasoning: A Solution for the HD-EPIC VQA Challenge
  3. 2026Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs
  4. 2025MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videos
  5. 2025JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation
  6. 2026OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models
  7. 2026Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models
  8. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMs