ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a promising route to long-video understanding. However, existing native-RL methods dispatch tool calls sequentially (i.e., one per turn): a single wrong crop propagates errors without peer correction, multi-turn tool calls corrupt context, and inference cost scales linearly with the number of turns. We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance. Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling. A cross-model contrast on a weaker-prior LMM supports this claim: format stays stable but RL elicits zero tool calls, indicating that prior strength is the shared driver of both format collapse and tool exploration. We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it. Across six long-video understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64. As tool capabilities become increasingly internalized in modern LMMs, RL must cooperate with the resulting priors, and ParaVT offers a general recipe for agentic RL. Code, data, and model weights are publicly available.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 54 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
50 of 54 references.
Closest ancestors · 3
Same direction, newest first
- 2026Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL30 Apr
- 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
Other references · 47
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
- 2017TALL: Temporal Activity Localization via Language Querycited by 581
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
35 more
- 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
- 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
- 2025ToolRL: Reward is All Tool Learning Needscited by 342
- 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
- 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
- 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
- 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 131
- 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 94
- 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
- 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 86
- 2025Scaling RL to Long Videoscited by 83
- 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
- 2025LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agentscited by 58
- 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
- 2025OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipecited by 37
- 2025Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuningcited by 33
- 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learningcited by 32
- 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
- 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoningcited by 24
- 2025LongVideoAgent: Multi-Agent Reasoning with Long Videoscited by 23
- 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 22
- 2026Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoningcited by 20
- 2026Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMscited by 20
- 2025Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidencecited by 17
- 2024Revisiting the Superficial Alignment Hypothesiscited by 12
- 2025MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Groundingcited by 11
- 2025Enhancing Agentic RL with Progressive Reward Shaping and Value-based Sampling Policy Optimizationcited by 9
- 2025Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-incited by 8
- 2025VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learningcited by 6
- 2025SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learningcited by 6
- 2025InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaborationcited by 5
- 2026SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaborationcited by 4
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understandingcited by 1
- 2026Video Evidence Indexing: Learning Where to Look from Video Previews for Token-Budgeted Long-Video Question Answeringcited by 0
- 2026VideoEvolve: Co-Evolving Memory and Retrieval for Long Video Understandingcited by 0
- 2026VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understandingcited by 0
- 2026OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioningcited by 0
- 2026OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning8 Oct 2026
- 2026VAMR: Multi-Question Agentic Reasoning for Efficient Long-Form Video Understanding8 Oct 2026
- 2026VideoEvolve: Co-Evolving Memory and Retrieval for Long Video Understanding7 Oct 2026
- 2026Video Evidence Indexing: Learning Where to Look from Video Previews for Token-Budgeted Long-Video Question Answering30 Sep 2026
- 2026StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understanding17 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026VC-Tooler: Learning Compositional and Adaptive Visual Tool Use
- 2026PyVision-RL: Forging Open Agentic Vision Models via RL
- 2026VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning
- 2026What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-Zoom
- 2025DeepEyesV2: Toward Agentic Multimodal Model
- 2026VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning
- 2026MetaForge: A Self-Evolving Multimodal Agent that Retrieves, Adapts, and Forges Tools On Demand ↗
- 2026Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models