VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning
On the map: LLM reasoning → Multimodal reasoning
also in 1 more
Abstract (arXiv)
Videos, with their unique temporal dimension, demand precise grounded understanding, where answers are directly linked to visual, interpretable evidence. Despite significant breakthroughs in text-based reasoning with large language models, multi-modal reasoning - especially for videos - remains limited. In this work, we fill this gap by introducing VideoMind, a novel video-language agent for temporal-grounded video reasoning. Our method involves two key innovations: (1) We identify four essential capabilities for grounded video reasoning and propose a role-based agentic workflow, comprising a planner to coordinate roles, a grounder for temporal event localization, a verifier to assess event candidates, and an answerer for question answering. (2) To efficiently integrate these roles during inference, we propose a novel Chain-of-LoRA mechanism, where a unified base model with multiple LoRA adapters is leveraged to enable seamless role switching, balancing efficiency and flexibility. Extensive experiments on 15 benchmarks across Grounded VideoQA, Video Temporal Grounding, and General VideoQA tasks demonstrate the effectiveness of the proposed scheme in advancing video agent, test-time scaling, and long-form video reasoning. Code, models, datasets, and demos are available at https://videomind.github.io/.
Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 86 of its 110 references that are in the map. First posted on arXiv 17 Mar 2025.
Builds on
86 of 110 references.
Closest ancestors · 8
Same direction, newest first
- 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
- 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos27 Jun
- 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning18 Feb
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action20 Mar
- 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
- 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
Other references · 78
- 2017Attention Is All You Needcited by 59,932
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Let's Verify Step by Stepcited by 3,598
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
38 more
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2023VideoChat: Chat-Centric Video Understandingcited by 851
- 2017Dense-Captioning Events in Videoscited by 789
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
- 2023VILA: On Pre-training for Visual Language Modelscited by 686
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
- 2017TALL: Temporal Activity Localization via Language Querycited by 581
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
- 2024Long Context Transfer from Language to Visioncited by 502
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 386
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
- 2024What matters when building vision-language models?cited by 321
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
- 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
- 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 283
- 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
- 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
- 2022Egocentric Video-Language Pretrainingcited by 231
- 2022Zero-Shot Video Question Answering via Frozen Bidirectional Language Modelscited by 229
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
- 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
- 2022Learning Video Representations from Large Language Modelscited by 204
- 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 200
- 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 189
- 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
- 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 172
- 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
50 most cited of 78 All 78 (JSON)
Built on by
Citing papers per year: 2025: 15; 2026: 26.
41 papers.
Sort:
By area:
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 46
- 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 39
- 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoningcited by 29
- 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 27
- 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
- 2025Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thoughtcited by 23
- 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 22
- 2026OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoningcited by 16
- 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 15
- 2025SiLVR: A Simple Language-based Video Reasoning Frameworkcited by 14
- 2025VIDEOP2R: Video Understanding from Perception to Reasoningcited by 8
- 2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Modelscited by 5
29 more
- 2025Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoningcited by 4
- 2025Video-Skill-CoT: Skill-based Chain-of-Thoughts for Domain-Adaptive Video Reasoningcited by 3
- 2025VideoPro: Adaptive Program Reasoning for Long Video Understandingcited by 3
- 2026Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Groundingcited by 3
- 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 3
- 2025Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understandingcited by 2
- 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigmscited by 2
- 2026MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understandingcited by 2
- 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understandingcited by 2
- 2026SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Groundingcited by 1
- 2026CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answeringcited by 1
- 2026EVIDENT: Routing MLLM Adaptation through Entity-Grounded Visual Evidence for Cross-Domain Video Temporal Groundingcited by 1
- 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challengescited by 1
- 2026Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Methodcited by 0
- 2026Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Modelscited by 0
- 2026Conditional Multi-Event Temporal Grounding in Long-Form Videocited by 0
- 2026TimeThink: Reasoning with Time for Video LLMscited by 0
- 2026Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agentscited by 0
- 2026Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMscited by 0
- 2026LAVE: Latent Visual Evidence-Enhanced Planning for Video Tool-use Agentscited by 0
- 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applicationscited by 0
- 2026Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervisioncited by 0
- 2026TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosiscited by 0
- 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agentscited by 0
- 2026VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understandingcited by 0
- 2026Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?cited by 0
- 2026STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Modelscited by 0
- 2026Agentic Video Understanding: A Surveycited by 0
- 2026VideoEvolve: Evolving Agent Harnesses for Video Temporal Groundingcited by 0
- 2026VideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding1 Oct 2026
- 2026Agentic Video Understanding: A Survey21 Sep 2026
- 2026Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?15 Sep 2026
- 2026VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding14 Sep 2026
- 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents11 Sep 2026
- 2026TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosis2 Sep 2026
- 2026STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Models28 Aug 2026
- 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding23 Aug 2026
- 2026Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervision21 Aug 2026
- 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges18 Aug 2026
- 2026Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs6 Aug 2026
- 2026LAVE: Latent Visual Evidence-Enhanced Planning for Video Tool-use Agents5 Aug 2026
29 more
- 2026MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding5 Aug 2026
- 2026Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents13 Jul 2026
- 2026TimeThink: Reasoning with Time for Video LLMs6 Jul 2026
- 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications28 Jun 2026
- 2026Conditional Multi-Event Temporal Grounding in Long-Form Video13 Jun 2026
- 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMs5 Jun 2026
- 2026EVIDENT: Routing MLLM Adaptation through Entity-Grounded Visual Evidence for Cross-Domain Video Temporal Grounding25 May 2026
- 2026CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering18 May 2026
- 2026Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models9 May 2026
- 2026OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning18 Apr 2026
- 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms10 Apr 2026
- 2026Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding9 Apr 2026
- 2026SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Grounding26 Mar 2026
- 2026Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Method15 Jan 2026
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs16 Dec 2025
- 2025Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understanding30 Nov 2025
- 2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Models23 Nov 2025
- 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understanding17 Nov 2025
- 2025VIDEOP2R: Video Understanding from Perception to Reasoning14 Nov 2025
- 2025Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoning17 Oct 2025
- 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning29 Sep 2025
- 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting29 Sep 2025
- 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis28 Sep 2025
- 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoning22 Sep 2025
- 2025VideoPro: Adaptive Program Reasoning for Long Video Understanding22 Sep 2025
- 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoning9 Jul 2025
- 2025Video-Skill-CoT: Skill-based Chain-of-Thoughts for Domain-Adaptive Video Reasoning4 Jun 2025
- 2025SiLVR: A Simple Language-based Video Reasoning Framework30 May 2025
- 2025Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought26 May 2025
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 46
- 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 39
- 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoningcited by 29
- 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 27
- 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
- 2025Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thoughtcited by 23
- 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 22
- 2026OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoningcited by 16
- 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 15
- 2025SiLVR: A Simple Language-based Video Reasoning Frameworkcited by 14
- 2025VIDEOP2R: Video Understanding from Perception to Reasoningcited by 8
- 2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Modelscited by 5
12 most cited of 23 All 23 (JSON)
- 2026Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?15 Sep 2026
- 2026VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding14 Sep 2026
- 2026Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervision21 Aug 2026
- 2026Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs6 Aug 2026
- 2026TimeThink: Reasoning with Time for Video LLMs6 Jul 2026
- 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMs5 Jun 2026
- 2026OASIS: On-Demand Hierarchical Event Memory for Streaming Video Reasoning18 Apr 2026
- 2026Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Method15 Jan 2026
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs16 Dec 2025
- 2025Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understanding30 Nov 2025
- 2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Models23 Nov 2025
- 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understanding17 Nov 2025
12 newest of 23 All 23 (JSON)
- 2026Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Groundingcited by 3
- 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applicationscited by 0
- 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agentscited by 0
- 2026Agentic Video Understanding: A Surveycited by 0
- 2026VideoEvolve: Evolving Agent Harnesses for Video Temporal Groundingcited by 0
- 2026VideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding1 Oct 2026
- 2026Agentic Video Understanding: A Survey21 Sep 2026
- 2026Behavior Quotient Learning for Low-Rank Adaptation of LLM Agents11 Sep 2026
- 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications28 Jun 2026
- 2026Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding9 Apr 2026
- 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigmscited by 2
- 2026SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Groundingcited by 1
- 2026Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Modelscited by 0
- 2026Conditional Multi-Event Temporal Grounding in Long-Form Videocited by 0
- 2026STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Modelscited by 0
- 2026STRAND: Benchmarking and Improving Object-Centric Spatio-Temporal Monitoring in Video Large Language Models28 Aug 2026
- 2026Conditional Multi-Event Temporal Grounding in Long-Form Video13 Jun 2026
- 2026Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models9 May 2026
- 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms10 Apr 2026
- 2026SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Grounding26 Mar 2026
- 2026MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understandingcited by 2
- 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understandingcited by 2
- 2026CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answeringcited by 1
- 2026EVIDENT: Routing MLLM Adaptation through Entity-Grounded Visual Evidence for Cross-Domain Video Temporal Groundingcited by 1
- 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challengescited by 1
- 2026Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agentscited by 0
- 2026LAVE: Latent Visual Evidence-Enhanced Planning for Video Tool-use Agentscited by 0
- 2026TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosiscited by 0
- 2026TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosis2 Sep 2026
- 2026TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding23 Aug 2026
- 2026Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges18 Aug 2026
- 2026LAVE: Latent Visual Evidence-Enhanced Planning for Video Tool-use Agents5 Aug 2026
- 2026MetaVideoAgent: Automated Video-Agent Evolution for Long-Form Video Understanding5 Aug 2026
- 2026Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents13 Jul 2026
- 2026EVIDENT: Routing MLLM Adaptation through Entity-Grounded Visual Evidence for Cross-Domain Video Temporal Grounding25 May 2026
- 2026CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering18 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025MUPA: Towards Multi-Path Agentic Reasoning for Grounded Video Question Answering
- 2026TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning
- 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking
- 2024Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillation
- 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks
- 2025ViQAgent: Zero-Shot Video Question Answering via Agent with Open-Vocabulary Grounding Validation
- 2025AVATAAR: Agentic Video Answering via Temporal Adaptive Alignment and Reasoning
- 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning