mlsanityData to 7 Oct 2026

AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs

arXiv 2606.07643 ↗First posted 1 Jun 2026cs.CV cs.AI cs.SDcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Recent advances in Omni-Multimodal Large Language Models (Omni-MLLMs) have enabled strong integration of vision, audio, and language. However, their audio-visual intelligence (AVI) remains insufficiently evaluated due to the lack of systematic and comprehensive benchmarks. We introduce AVI-Bench, a cognitively inspired benchmark that evaluates Omni-MLLMs across three stages, perception, understanding, and reasoning, through cross-modal tasks requiring joint audio-visual interpretation. This design enables fine-grained diagnosis of model capabilities and failure modes. To further assess robustness beyond familiar domains, we propose AVI-Bench-PriSe, an extension that probes models' primitive audio-visual sensation using unfamiliar, low-semantic stimuli, testing generalization beyond common training distributions. Extensive experiments on both open-source and closed-source models reveal substantial limitations in current Omni-MLLMs. Based on these findings, we present a four-level AVI taxonomy. Overall, AVI-Bench provides a principled evaluation framework to guide the development of more robust and generalizable AVI. Project website: https://fudancvl.github.io/AVI-Bench/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 56 of its 72 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

56 of 72 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities23 May
  2. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs6 Feb
  3. 2024AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?3 Dec
  4. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 52

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2024Qwen2.5 Technical Reportcited by 4,713
  8. 2024GPT-4o System Cardcited by 4,467
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  12. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
38 more
  1. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  3. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  4. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  5. 2025Qwen2.5-Omni Technical Reportcited by 704
  6. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  7. 2024Qwen2-Audio Technical Reportcited by 543
  8. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  9. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  10. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 330
  11. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
  12. 2022What does a platypus look like? Generating customized prompts for zero-shot image classificationcited by 263
  13. 2023Pengi: An Audio Language Model for Audio Taskscited by 217
  14. 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 213
  15. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  16. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  17. 2021Localizing Visual Sounds the Hard Waycited by 169
  18. 2023VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Datasetcited by 168
  19. 2022Audio-Visual Segmentationcited by 148
  20. 2023VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Datasetcited by 138
  21. 2021Towards artificial general intelligence via a multimodal foundation modelcited by 112
  22. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  23. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
  24. 2024video-SALMONN: Speech-Enhanced Audio-Visual Large Language Modelscited by 101
  25. 2025R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learningcited by 99
  26. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  27. 2021Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videoscited by 86
  28. 2022Localizing Visual Sounds the Easy Waycited by 81
  29. 2025Ola: Pushing the Frontiers of Omni-Modal Language Modelcited by 79
  30. 2025HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understandingcited by 58
  31. 2022I2MVFormer: Large Language Model Generated Multi-View Document Supervision for Zero-Shot Image Classificationcited by 54
  32. 2024AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Modelscited by 50
  33. 2023Prompting Segmentation with Sound Is Generalizable Audio-Visual Source Localizercited by 35
  34. 2024Ref-AVS: Refer and Segment Objects in Audio-Visual Scenescited by 32
  35. 2023Audio-Visual Instance Segmentationcited by 21
  36. 2025On Path to Multimodal Generalist: General-Level and General-Benchcited by 18
  37. 2024WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoningcited by 16
  38. 2024Can Textual Semantics Mitigate Sounding Object Segmentation Preference?cited by 13

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Audio-Visual Intelligence in Large Foundation Models
  2. 2026Do Audio-Visual Large Language Models Really See and Hear?
  3. 2025JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation
  4. 2026AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models
  5. 2026When Vision Speaks for Sound
  6. 2025Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception
  7. 2025AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning
  8. 2026Crab+: A Scalable and Unified Audio-Visual Scene Understanding Model with Explicit Cooperation