A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
also in 1 more
Abstract (arXiv)
Multimodal Large Language Models have achieved strong performance in single-video understanding, yet their ability to reason across multiple videos remains limited. Existing approaches typically concatenate multiple videos into a single input and perform direct inference, which introduces training-inference mismatch, information loss from frame compression, and a lack of explicit cross-video coordination. Meanwhile, current multi-video benchmarks primarily emphasize event-level comparison, leaving identity-level matching, fine-grained discrimination, and structured multi-step reasoning underexplored. To address these gaps, we introduce MVX-Bench, a Multi-Video Cross-Dimension Benchmark that reformulates 11 classical computer vision tasks into a unified multi-video question-answering framework, comprising 1,442 questions over 4,255 videos from diverse real-world datasets. We further propose SAMA, a Skill-Augmented Agentic Framework for Multi-Video Understanding, which integrates visual tools, task-specific skills, and a conflict-aware verification mechanism to enable iterative and structured reasoning. Experimental results show that SAMA outperforms strong open-source baselines and GPT on MVX-Bench, and ablations validate the effectiveness of skill design and conflict resolution.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 45 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
30 of 45 references.
Closest ancestors · 8
Same direction, newest first
- 2025MVU-Eval: Towards Multi-Video Understanding Evaluation for Multimodal LLMs10 Nov
- 2025CVBench: Benchmarking Cross-Video Synergies for Complex Multimodal Reasoning27 Aug
- 2025Can Video Large Multimodal Models Think Like Doubters-or Double-Down: A Study on Defeasible Video Entailment27 Jun
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding18 Mar
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun
Other references · 22
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
- 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
- 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
- 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
- 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
10 more
- 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 207
- 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
- 2025VideoAgent2: Enhancing the LLM-Based Agent System for Long-Form Video Understanding by Uncertainty-Aware CoTcited by 36
- 2025Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMscited by 27
- 2018FIVR: Fine-grained Incident Video Retrievalcited by 26
- 2025Open-World Object Counting in Videoscited by 11
- 2023ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videoscited by 8
- 2024ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of Videocited by 6
- 2025ConViS-Bench: Estimating Video Similarity Through Semantic Conceptscited by 4
- 2025CI-VID: A Coherent Interleaved Text-Video Datasetcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understanding
- 2026Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning
- 2025CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models
- 2025Agentic Video Intelligence: A Flexible Framework for Advanced Video Exploration and Understanding
- 2026AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents
- 2026Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction
- 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaboration
- 2025Reasoning is All You Need for Video Generalization: A Counterfactual Benchmark with Sub-question Evaluation