mlsanityData to 7 Oct 2026

VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning

arXiv 2503.13444 ↗First posted 17 Mar 2025cs.CV cs.AIcited by 41

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Videos, with their unique temporal dimension, demand precise grounded understanding, where answers are directly linked to visual, interpretable evidence. Despite significant breakthroughs in text-based reasoning with large language models, multi-modal reasoning - especially for videos - remains limited. In this work, we fill this gap by introducing VideoMind, a novel video-language agent for temporal-grounded video reasoning. Our method involves two key innovations: (1) We identify four essential capabilities for grounded video reasoning and propose a role-based agentic workflow, comprising a planner to coordinate roles, a grounder for temporal event localization, a verifier to assess event candidates, and an answerer for question answering. (2) To efficiently integrate these roles during inference, we propose a novel Chain-of-LoRA mechanism, where a unified base model with multiple LoRA adapters is leveraged to enable seamless role switching, balancing efficiency and flexibility. Extensive experiments on 15 benchmarks across Grounded VideoQA, Video Temporal Grounding, and General VideoQA tasks demonstrate the effectiveness of the proposed scheme in advancing video agent, test-time scaling, and long-form video reasoning. Code, models, datasets, and demos are available at https://videomind.github.io/.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 86 of its 110 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

86 of 110 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
  2. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  3. 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos27 Jun
  4. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning18 Feb
  5. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action20 Mar
  6. 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
  7. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
  8. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep

Other references · 78

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
38 more
  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  4. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  5. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  6. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  7. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  8. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  9. 2017Dense-Captioning Events in Videoscited by 789
  10. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  11. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  12. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  13. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  14. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  15. 2024Long Context Transfer from Language to Visioncited by 502
  16. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
  17. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  18. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
  19. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  20. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
  21. 2024What matters when building vision-language models?cited by 321
  22. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
  23. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
  24. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 283
  25. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
  26. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
  27. 2022Egocentric Video-Language Pretrainingcited by 231
  28. 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
  29. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
  30. 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
  31. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  32. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  33. 2022Learning Video Representations from Large Language Modelscited by 204
  34. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
  35. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 189
  36. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  37. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 172
  38. 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 26.

41 papers.

Sort:

By area:

  1. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 46
  2. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 39
  3. 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoningcited by 29
  4. 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 27
  5. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
  6. 2025Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thoughtcited by 23
  7. 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 22
  8. 2026OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoningcited by 16
  9. 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 15
  10. 2025SiLVR: A Simple Language-based Video Reasoning Frameworkcited by 14
  11. 2025VIDEOP2R: Video Understanding from Perception to Reasoningcited by 8
  12. 2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Modelscited by 5
29 more
  1. 2025Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoningcited by 4
  2. 2025Video-Skill-CoT: Skill-based Chain-of-Thoughts for Domain-Adaptive Video Reasoningcited by 3
  3. 2025VideoPro: Adaptive Program Reasoning for Long Video Understandingcited by 3
  4. 2026Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Groundingcited by 3
  5. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 3
  6. 2025Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understandingcited by 2
  7. 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigmscited by 2
  8. 2026MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understandingcited by 2
  9. 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understandingcited by 2
  10. 2026SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Groundingcited by 1
  11. 2026CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answeringcited by 1
  12. 2026EVIDENT: Routing MLLM Adaptation through Entity-Grounded Visual Evidence for Cross-Domain Video Temporal Groundingcited by 1
  13. 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challengescited by 1
  14. 2026Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Methodcited by 0
  15. 2026Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Modelscited by 0
  16. 2026Conditional Multi-Event Temporal Grounding in Long-Form Videocited by 0
  17. 2026TimeThink: Reasoning with Time for Video LLMscited by 0
  18. 2026Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agentscited by 0
  19. 2026Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMscited by 0
  20. 2026LAVE: Latent Visual Evidence-Enhanced Planning for Video Tool-use Agentscited by 0
  21. 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applicationscited by 0
  22. 2026Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervisioncited by 0
  23. 2026TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosiscited by 0
  24. 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agentscited by 0
  25. 2026VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understandingcited by 0
  26. 2026Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?cited by 0
  27. 2026STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Modelscited by 0
  28. 2026Agentic Video Understanding: A Surveycited by 0
  29. 2026VideoEvolve: Evolving Agent Harnesses for Video Temporal Groundingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering
  2. 2026TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning
  3. 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking
  4. 2024Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillation
  5. 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks
  6. 2025ViQAgent: Zero-Shot Video Question Answering via Agent with Open-Vocabulary Grounding Validation
  7. 2025AVATAAR: Agentic Video Answering via Temporal Adaptive Alignment and Reasoning
  8. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning