mlsanityData to 7 Oct 2026

ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning

arXiv 2605.20342 ↗First posted 19 May 2026cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Training large multimodal models (LMMs) via reinforcement learning (RL) to natively invoke video-processing tools (e.g., cropping) has become a promising route to long-video understanding. However, existing native-RL methods dispatch tool calls sequentially (i.e., one per turn): a single wrong crop propagates errors without peer correction, multi-turn tool calls corrupt context, and inference cost scales linearly with the number of turns. We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling, dispatching multiple time-window crops in a single turn for cleaner context and better fault tolerance. Yet applying standard RL to ParaVT reveals an obstacle we term the Tool Prior Paradox: the pretrained tool priors that enable tool exploration also destabilize cold-started structural format and expose the skip-tool reward shortcut under temperature sampling. A cross-model contrast on a weaker-prior LMM supports this claim: format stays stable but RL elicits zero tool calls, indicating that prior strength is the shared driver of both format collapse and tool exploration. We propose PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO), which augments standard RL with two complementary mechanisms: (i) a targeted format reward applied only at the structural-token positions most prone to collapse, and (ii) a per-prompt frame-budget randomization that creates training prompts where calling the tool yields a measurable reward signal over skipping it. Across six long-video understanding benchmarks, ParaVT improves over the Qwen3-VL baseline by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64. As tool capabilities become increasingly internalized in modern LMMs, RL must cooperate with the resulting priors, and ParaVT offers a general recipe for agentic RL. Code, data, and model weights are publicly available.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 54 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

50 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL30 Apr
  2. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  3. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar

Other references · 47

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  8. 2025Qwen3-VL Technical Reportcited by 2,534
  9. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  11. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  12. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
35 more
  1. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  2. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  3. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 402
  4. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  5. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
  6. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 275
  7. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  8. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
  9. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  10. 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 131
  11. 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 94
  12. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
  13. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 86
  14. 2025Scaling RL to Long Videoscited by 83
  15. 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
  16. 2025LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agentscited by 58
  17. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
  18. 2025OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipecited by 37
  19. 2025Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuningcited by 33
  20. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learningcited by 32
  21. 2025LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoningcited by 27
  22. 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoningcited by 24
  23. 2025LongVideoAgent: Multi-Agent Reasoning with Long Videoscited by 23
  24. 2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 22
  25. 2026Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoningcited by 20
  26. 2026Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMscited by 20
  27. 2025Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidencecited by 17
  28. 2024Revisiting the Superficial Alignment Hypothesiscited by 12
  29. 2025MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Groundingcited by 11
  30. 2025Enhancing Agentic RL with Progressive Reward Shaping and Value-based Sampling Policy Optimizationcited by 9
  31. 2025Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-incited by 8
  32. 2025VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learningcited by 6
  33. 2025SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learningcited by 6
  34. 2025InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaborationcited by 5
  35. 2026SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaborationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026VC-Tooler: Learning Compositional and Adaptive Visual Tool Use
  2. 2026PyVision-RL: Forging Open Agentic Vision Models via RL
  3. 2026VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning
  4. 2026What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-Zoom
  5. 2025DeepEyesV2: Toward Agentic Multimodal Model
  6. 2026VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning
  7. 2026MetaForge: A Self-Evolving Multimodal Agent that Retrieves, Adapts, and Forges Tools On Demand ↗
  8. 2026Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models