ReTool-Video: Recursive Tool-Using Video Agents with Meta-Augmented Tool Grounding
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Video understanding requires active evidence seeking, motivating tool-augmented video agents for temporal reasoning, cross-modal understanding, and complex question answering. Existing video agents have improved video reasoning with retrieval, memory, frame inspection, and verifier tools, but they still face two limitations: (1) a coarse tool space that lacks fine-grained operations for compositional reasoning; and (2) a flat action space that forces high-level video intents into primitive executable tool calls. In this paper, we address these challenges with two complementary designs. First, we construct a MetaAug-Video Tool Library (MVTL), an extensible tool library with 134 registered tools, including 26 base tools for general multimodal signal processing and 108 meta tools for filtering, aggregation, reranking, formatting, and other intermediate-result operations. MVTL supports dual-level access to both structured video information and raw modal evidence, enabling diverse video reasoning scenarios. Second, we propose ReTool-Video, a recursive tool-using method that grounds high-level video intents into executable tool chains. In ReTool-Video, matched actions are executed directly, while unmatched intents are delegated to a resolver for parameter repair, tool substitution, or decomposition. This allows abstract actions such as temporal merging, cross-modal verification, or repeated-event aggregation to be progressively translated into concrete multimodal operations at runtime. Experiments on MVBench, MLVU, and Video-MME w/o sub. show that ReTool-Video consistently outperforms strong baselines. Further analysis demonstrates that recursive grounding and fine-grained meta tools improve the stability and effectiveness of complex video understanding.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 78 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
46 of 78 references.
Closest ancestors · 7
Same direction, newest first
- 2026VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning22 Jan
- 2025VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding13 Dec
- 2025Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task11 Dec
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
- 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning6 Aug
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action20 Mar
- 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
Other references · 39
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2024GPT-4o System Cardcited by 4,588
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
- 2025OpenAI GPT-5 System Cardcited by 989
27 more
- 2017Dense-Captioning Events in Videoscited by 793
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
- 2024Long Context Transfer from Language to Visioncited by 515
- 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
- 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
- 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
- 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 134
- 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107
- 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 98
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
- 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 68
- 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 65
- 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 58
- 2025PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PCcited by 40
- 2024Agent-Oriented Planning in Multi-Agent Systemscited by 36
- 2025VideoExplorer: Think With Videos For Agentic Long-Video Understandingcited by 29
- 2025FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learningcited by 21
- 2026Agentic Very Long Video Understandingcited by 20
- 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seekingcited by 17
- 2025ReCode: Unify Plan and Action for Universal Granularity Controlcited by 15
- 2025GAP: Graph-Based Agent Planning with Parallel Tool Use and Reinforcement Learningcited by 11
- 2025CFVBench: A Comprehensive Video Benchmark for Fine-grained Multimodal Retrieval-Augmented Generationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026VideoResearcher: Self-Improving Tool Design for Long-Video Understanding
- 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning
- 2026VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning
- 2026Agentic Video Understanding: A Survey
- 2026VideoSearch-R1: Iterative Video Retrieval and Reasoning via Soft Query Refinement
- 2025CAViAR: Critic-Augmented Video Agentic Reasoning
- 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis
- 2026A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding