mlsanityData to 7 Oct 2026

QMAVIS: Long Video-Audio Understanding using Fusion of Large Multimodal Models

arXiv 2601.06573 ↗First posted 10 Jan 2026cs.AI cs.MMcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Large Multimodal Models (LMMs) for video-audio understanding have traditionally been evaluated only on shorter videos of a few minutes long. In this paper, we introduce QMAVIS (Q Team-Multimodal Audio Video Intelligent Sensemaking), a novel long video-audio understanding pipeline built through a late fusion of LMMs, Large Language Models, and speech recognition models. QMAVIS addresses the gap in long-form video analytics, particularly for longer videos of a few minutes to beyond an hour long, opening up new potential applications in sensemaking, video content analysis, embodied AI, etc. Quantitative experiments using QMAVIS demonstrated a 38.75% improvement over state-of-the-art video-audio LMMs like VideoLlaMA2 and InternVL2 on the VideoMME (with subtitles) dataset, which comprises long videos with audio information. Evaluations on other challenging video understanding datasets like PerceptionTest and EgoSchema saw up to 2% improvement, indicating competitive performance. Qualitative experiments also showed that QMAVIS is able to extract the nuances of different scenes in a long video audio content while understanding the overarching narrative. Ablation studies were also conducted to ascertain the impact of each component in the fusion pipeline.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 14 references that are in the map. First posted on arXiv 10 Jan 2026.

Builds on

12 of 14 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
  2. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
  3. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 9

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  7. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  8. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 330
  9. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Audio-centric Video Understanding Benchmark without Text Shortcut
  2. 2023Audio-Visual LLM for Video Understanding
  3. 2024Visual Context Window Extension: A New Perspective for Long Video Understanding
  4. 2025See, Hear, and Understand: Benchmarking Audiovisual Human Speech Understanding in Multimodal Large Language Models
  5. 2025Unleashing Hour-Scale Video Training for Long Video-Language Understanding
  6. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
  7. 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understanding
  8. 2026Do Audio-Visual Large Language Models Really See and Hear?