mlsanityData to 14 Oct 2026

LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agents

arXiv 2503.10200 ↗First posted 13 Mar 2025cs.CVcited by 58

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Existing MLLMs encounter significant challenges in modeling the temporal context within long videos. Currently, mainstream Agent-based methods use external tools to assist a single MLLM in answering long video questions. Despite such tool-based support, a solitary MLLM still offers only a partial understanding of long videos, resulting in limited performance. In order to better address long video tasks, we introduce LVAgent, the first framework enabling multi-round dynamic collaboration of MLLM agents in long video understanding. Our method consists of four key steps: 1) Selection: We pre-select appropriate agents from the model library to form optimal agent teams based on different tasks. 2) Perception: We design an effective retrieval scheme for long videos to improve the coverage of critical temporal segments while maintaining computational efficiency. 3) Action: Agents answer long video questions and exchange reasons. 4) Reflection: We evaluate each agent's performance in each round of discussion and optimize the agent team for dynamic collaboration. The agents iteratively refine their answers by multi-round dynamical collaboration of MLLM agents. LVAgent is the first agent system method that outperforms all closed-source models (like GPT-4o) and open-source models (like InternVL-2.5 and Qwen2-VL) in the long video understanding tasks. Our LVAgent achieves an accuracy of 80% on four mainstream long video understanding tasks. Notably, LVAgent improves accuracy by 13.3% on LongVideoBench. Code is available at https://github.com/64327069/LVAgent.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 51 of its 75 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

51 of 75 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024VCA: Video Curious Agent for Long Video Understanding12 Dec
  2. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding18 Mar
  3. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agent15 Mar

Other references · 48

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  8. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  9. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  10. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  11. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  12. 2023VideoChat: Chat-Centric Video Understandingcited by 857
36 more
  1. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  2. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  3. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  4. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
  5. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  6. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  7. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  8. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  9. 2024Long Context Transfer from Language to Visioncited by 515
  10. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  11. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
  12. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videoscited by 309
  13. 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 300
  14. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 290
  15. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  16. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 229
  17. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 225
  18. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 213
  19. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 159
  20. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 153
  21. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 147
  22. 2024Aria: An Open Multimodal Native Mixture-of-Experts Modelcited by 138
  23. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 138
  24. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 137
  25. 2021Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Losscited by 116
  26. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 106
  27. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecturecited by 102
  28. 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 98
  29. 2020Multimodal Pretraining for Dense Video Captioningcited by 79
  30. 2025VideoRAG: Retrieval-Augmented Generation over Video Corpuscited by 57
  31. 2024VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Modelscited by 41
  32. 2024LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaborationcited by 38
  33. 2024SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledgecited by 20
  34. 2022An Empirical Study on Activity Recognition in Long Surgical Videoscited by 13
  35. 2024Percept, Chat, and then Adapt: Multimodal Knowledge Transfer of Foundation Models for Open-World Video Recognitioncited by 6
  36. 2022Low-Resolution Action Recognition for Tiny Actions Challengecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 44.

58 papers.

Sort:

By area:

  1. 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 33
  2. 2025VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoningcited by 32
  3. 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understandingcited by 29
  4. 2025VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understandingcited by 25
  5. 2026Think3D: Thinking with Space for Spatial Reasoningcited by 24
  6. 2025UniFlow: A Unified Pixel Flow Tokenizer for Visual Understanding and Generationcited by 23
  7. 2025FlexSelect: Flexible Token Selection for Efficient Long Video Understandingcited by 18
  8. 2026Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoningcited by 14
  9. 2026Refer-Agent: A Collaborative Multi-Agent System with Reasoning and Reflection for Referring Video Object Segmentationcited by 9
  10. 2026Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understandingcited by 9
  11. 2026Hierarchical Long Video Understanding with Audiovisual Entity Cohesion and Agentic Searchcited by 8
  12. 2026Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baselinecited by 8
38 more
  1. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  2. 2025Debunk and Infer: Multimodal Fake News Detection via Diffusion-Generated Evidence and LLM Reasoningcited by 7
  3. 2025VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learningcited by 6
  4. 2025SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learningcited by 6
  5. 2026ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learningcited by 6
  6. 2025Super Encoding Network: Recursive Association of Multi-Modal Encoders for Video Understandingcited by 4
  7. 2025A Neurosymbolic Agent System for Compositional Visual Reasoningcited by 4
  8. 2025GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memorycited by 4
  9. 2026HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answeringcited by 4
  10. 2026FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognitioncited by 4
  11. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4
  12. 2026Native Active Perception as Reasoning for Omni-Modal Understandingcited by 4
  13. 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agentscited by 3
  14. 2026VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoningcited by 3
  15. 2026Video-Oasis: Rethinking Evaluation of Video Understandingcited by 3
  16. 2026Visual Agentic Memory: Enabling Online Long Video Understanding via Online Indexing, Hierarchical Memory, and Agentic Retrievalcited by 3
  17. 2026Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoningcited by 3
  18. 2025G-UBS: Towards Robust Understanding of Implicit Feedback via Group-Aware User Behavior Simulationcited by 2
  19. 2025Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understandingcited by 2
  20. 2026A Multi-Agent Perception-Action Alliance for Efficient Long Video Reasoningcited by 2
  21. 2026AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understandingcited by 2
  22. 2026Routing Before Looking: Query-Adaptive Evidence Acquisition for Long-form Video Understandingcited by 2
  23. 2026DepthAgent: Towards Better Universal Depth Estimation via Sample-wise Expert Selectioncited by 1
  24. 2026Skill-3D: Evolving Scene-Aware Skills for Agentic 3D Spatial Reasoningcited by 1
  25. 2026MAVIS: Multi-Agent Video Retrieval via Structured Video Understandingcited by 1
  26. 2026CoCoSI: Collaborative Cognitive Map Construction for Spatial Intelligencecited by 1
  27. 2026Rethinking RAG in Long Videos: What to Retrieve and How to Use It?cited by 1
  28. 2026MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMscited by 1
  29. 2026REVEAL: A Rubric-Guided Agent for Explicit Evidence Sufficiency Verificationin Long-Video Question Answeringcited by 1
  30. 2026DIVER: Dynamic Iterative Visual Evidence Reasoning for Multimodal Fake News Detectioncited by 0
  31. 2026Vision Verification Enhanced Fusion of VLMs for Efficient Visual Reasoningcited by 0
  32. 2026LLM-MRD: LLM-Guided Multi-View Reasoning Distillation for Fake News Detectioncited by 0
  33. 2026GOPAgen: Codec-Aware Agentic Long-Video Understanding with Structured Memorycited by 0
  34. 2026From Content to Knowledge: Lightning Fast Long-Video Understanding with Neural Knowledge Representationscited by 0
  35. 2026HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoningcited by 0
  36. 2026Accuracy and Cost Claims Do Not Survive Re-Execution in Agentic VideoQAcited by 0
  37. 2026QSVideo: Query-Conditioned Semantic Temporal Retrieval for Video Understandingcited by 0
  38. 2026GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMscited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaboration
  2. 2024Adaptive Video Understanding Agent: Enhancing efficiency with dynamic frame sampling and feedback-driven reasoning
  3. 2025LongVideoAgent: Multi-Agent Reasoning with Long Videos
  4. 2026VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understanding
  5. 2026Think, Then Verify: A Hypothesis-Verification Multi-Agent Framework for Long Video Understanding
  6. 2026HiCrew: Hierarchical Reasoning for Long-Form Video Understanding via Question-Aware Multi-Agent Collaboration
  7. 2024TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering
  8. 2026VideoSEAL: Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authority