mlsanityData to 7 Oct 2026

OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs

arXiv 2510.10689 ↗First posted 12 Oct 2025cs.AIcited by 54

On the map: LLM reasoning → Audio-language model reasoning

also in 1 more
Abstract (arXiv)

Recent advances in multimodal large language models (MLLMs) have demonstrated substantial potential in video understanding. However, existing benchmarks fail to comprehensively evaluate synergistic reasoning capabilities across audio and visual modalities, often neglecting either one of the modalities or integrating them in a logically inconsistent manner. To bridge this gap, we introduce OmniVideoBench, a large-scale and rigorously designed benchmark dedicated to assessing synergistic audio-visual understanding, with a strong emphasis on modality complementarity and logical consistency. Specifically, OmniVideoBench comprises 1000 high-quality question-answer(QA) pairs, each annotated with step-by-step reasoning traces, derived from 628 diverse videos ranging from several seconds to 30 minutes, and manually verified to guarantee complete correctness and uniqueness. Moreover, OmniVideoBench encompasses 13 carefully designed question types, covering temporal reasoning, spatial localization, counting, causal inference, summarization, and beyond, thereby capturing the essential challenges of video understanding. Evaluation of multiple MLLMs on OmniVideoBench reveals a pronounced gap between model performance and human reasoning, with open-source models lagging significantly behind their closed-source counterparts, underscoring the inherent difficulty of genuine audio-visual reasoning. We will release OmniVideoBench to foster the development of MLLMs with stronger and more generalizable reasoning capabilities.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 59 references that are in the map. First posted on arXiv 12 Oct 2025.

Builds on

48 of 59 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities23 May
  2. 2025Qwen2.5-Omni Technical Report26 Mar
  3. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark24 Oct

Other references · 45

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  8. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  9. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  10. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  11. 2023A Survey on Multimodal Large Language Modelscited by 898
  12. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
33 more
  1. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  2. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  3. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  4. 2025Qwen3-Omni Technical Reportcited by 451
  5. 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 412
  6. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  7. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 191
  8. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  9. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 180
  10. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  11. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 147
  12. 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiencycited by 123
  13. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  14. 2024M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thoughtcited by 120
  15. 2025A Comprehensive Survey on Long Context Language Modelingcited by 113
  16. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
  17. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 102
  18. 2024OmniBench: Towards The Future of Universal Omni-Language Modelscited by 100
  19. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
  20. 2021VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluationcited by 95
  21. 2025Ola: Pushing the Frontiers of Omni-Modal Language Modelcited by 79
  22. 2025YuE: Scaling Open Foundation Models for Long-Form Music Generationcited by 68
  23. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  24. 2023Exploring Diverse In-Context Configurations for Image Captioningcited by 63
  25. 2025HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understandingcited by 58
  26. 2024MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilitiescited by 57
  27. 2024AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?cited by 46
  28. 2025Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?cited by 43
  29. 2024PCA-Bench: Evaluating Multimodal Large Language Models in Perception-Cognition-Action Chaincited by 29
  30. 2023MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteriacited by 27
  31. 2025AVTrustBench: Assessing and Enhancing Reliability and Robustness in Audio-Visual LLMscited by 19
  32. 2025DAVE: Diagnostic benchmark for Audio Visual Evaluationcited by 5
  33. 2025Engagement Prediction of Short Videos with Large Multimodal Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 44.

54 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2026OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Modelscited by 23
  3. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 22
  4. 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Datacited by 21
  5. 2025OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Modelscited by 21
  6. 2026OmniGAIA: Towards Native Omni-Modal AI Agentscited by 21
  7. 2026MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videoscited by 14
  8. 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
  9. 2026SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Modelscited by 12
  10. 2025MVU-Eval: Towards Multi-Video Understanding Evaluation for Multimodal LLMscited by 9
  11. 2026OmniVideo-R1: Reinforcing Audio-visual Reasoning with Query Intention and Modality Attentioncited by 9
  12. 2025MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialoguescited by 7
38 more
  1. 2025IF-VidCap: Can Video Caption Models Follow Instructions?cited by 7
  2. 2026Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Groundingcited by 6
  3. 2025FysicsWorld: A Unified Full-Modality Benchmark for Any-to-Any Understanding, Generation, and Reasoningcited by 5
  4. 2025A Benchmark for Omni-Modal Reasoning in Long Videoscited by 5
  5. 2026OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answeringcited by 5
  6. 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoningcited by 5
  7. 2026OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chainscited by 5
  8. 2026SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understandingcited by 3
  9. 2026Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMscited by 3
  10. 2026OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Modelscited by 3
  11. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 3
  12. 2026Native Active Perception as Reasoning for Omni-Modal Understandingcited by 3
  13. 2026OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Modelscited by 3
  14. 2025ViDiC: Video Difference Captioningcited by 2
  15. 2026VEBench:Benchmarking Large Multimodal Models for Real-World Video Editingcited by 2
  16. 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoningcited by 2
  17. 2026Sandboxed Coding Agents are Competitive Omni-modal Task Solverscited by 2
  18. 2026AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compressioncited by 2
  19. 2026OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Modelscited by 2
  20. 2026TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streamingcited by 2
  21. 2026Realtime-Venus: A full-duplex interaction system with asynchronous delegationcited by 2
  22. 2026Qwen3.8-Omni: Towards Native Omni-Modal Agentscited by 2
  23. 2026TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videoscited by 1
  24. 2026VideoOdyssey: A Benchmark for Ultra-Long-Context and Omni-Modal Video Understandingcited by 1
  25. 2026From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMscited by 1
  26. 2026OmniReasoner: Thinking with Long Audio-Video via Native Tool Usecited by 1
  27. 2026OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Modelscited by 1
  28. 2026AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilitiescited by 1
  29. 2026Omni-Streaming Thinkingcited by 1
  30. 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reorderingcited by 0
  31. 2026CogniRoute: Learning to Route Social Evidence in Omni-Modal Modelscited by 0
  32. 2026video-SALMONN-R3: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understandingcited by 0
  33. 2026Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Modelscited by 0
  34. 2026OmniAssistBench: Assistant-style Interaction Benchmark for Omni-LLMscited by 0
  35. 2026OmniKVQuant: KV Cache Quantization for Omni-LLMscited by 0
  36. 2026Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?cited by 0
  37. 2026Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Modelcited by 0
  38. 2026OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialoguecited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation
  2. 2026OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
  3. 2025See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Models
  4. 2025AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning
  5. 2025CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models
  6. 2026AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models
  7. 2026Empowering Long-form Omni-modal Understanding with Robust Audio Perception
  8. 2025Audio-centric Video Understanding Benchmark without Text Shortcut