VidBridge-R1: Bridging QA and Captioning for RL-based Video Understanding Models with Intermediate Proxy Tasks
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
The "Reason-Then-Respond" paradigm, enhanced by Reinforcement Learning, has shown great promise in advancing Multimodal Large Language Models. However, its application to the video domain has led to specialized models that excel at either question answering (QA) or captioning tasks, but struggle to master both. Naively combining reward signals from these tasks results in mutual performance degradation, which we attribute to a conflict between their opposing task natures. To address this challenge, we propose a novel training framework built upon two intermediate proxy tasks: DarkEventInfer, which presents videos with masked event segments, requiring models to infer the obscured content based on contextual video cues; and MixVidQA, which presents interleaved video sequences composed of two distinct clips, challenging models to isolate and reason about one while disregarding the other. These proxy tasks compel the model to simultaneously develop both holistic, divergent understanding and precise, convergent reasoning capabilities. Embodying this framework, we present VidBridge-R1, the first versatile video reasoning model that effectively bridges the paradigm conflict. Extensive experiments show that VidBridge-R1 achieves significant performance gains on both QA and captioning within one model, demonstrating the efficacy of our approach in fostering more generalizable and powerful video understanding models.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 48 of its 58 references that are in the map. First posted on arXiv 10 Jun 2025.
Builds on
48 of 58 references.
Closest ancestors · 13
Same direction, newest first
- 2025VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinking2 Jun
- 2025Reinforcing Video Reasoning with Focused Thinking30 May
- 2025Mavors: Multi-granularity Video Representation for Multimodal Large Language Model14 Apr
- 2025TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning13 Apr
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2025Tarsier2: Advancing Large Vision-Language Models from Detailed Video Description to Comprehensive Video Understanding14 Jan
- 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
- 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecture4 Sep
- 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
Other references · 35
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Visual Instruction Tuningcited by 7,792
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2017The Kinetics Human Action Video Datasetcited by 1,894
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
23 more
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
- 2025Kimi-VL Technical Reportcited by 381
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
- 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 316
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
- 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 234
- 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 214
- 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
- 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 140
- 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 132
- 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 131
- 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
- 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 86
- 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 86
- 2025Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thoughtcited by 69
- 2025R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPOcited by 52
- 2024KVQ: Kwai Video Quality Assessment for Short-form Videoscited by 38
- 2025VidCapBench: A Comprehensive Benchmark of Video Captioning for Controllable Text-to-Video Generationcited by 22
- 2025VerIPO: Cultivating Long Reasoning in Video-LLMs via Verifier-Gudied Iterative Policy Optimizationcited by 11
- 2025Unlocking the Potential of Difficulty Prior in RL-based Multimodal Reasoningcited by 7
Built on by
Citing papers per year: 2025: 5; 2026: 5.
10 papers.
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
- 2025VIDEOP2R: Video Understanding from Perception to Reasoningcited by 8
- 2025Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Modelscited by 7
- 2026Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Designcited by 3
- 2025AVC-DPO: Aligned Video Captioning via Direct Preference Optimizationcited by 2
- 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
- 2026VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioningcited by 1
- 2026CapRiCorn-1K: A Comprehensive Benchmark for Video Captioning and Subject Referential Consistency Across Temporal Scalescited by 1
- 2026PercepCap: Video Captioner with Structured Spatio-Temporal Perceptioncited by 0
- 2026Beyond Sparse Rewards: A New Benchmark and Structure-Aware Graph Alignment for Micro-Drama Understandingcited by 0
- 2026Beyond Sparse Rewards: A New Benchmark and Structure-Aware Graph Alignment for Micro-Drama Understanding7 Sep 2026
- 2026PercepCap: Video Captioner with Structured Spatio-Temporal Perception22 Jul 2026
- 2026CapRiCorn-1K: A Comprehensive Benchmark for Video Captioning and Subject Referential Consistency Across Temporal Scales20 Jun 2026
- 2026VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning27 May 2026
- 2026Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design25 Feb 2026
- 2025Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models28 Nov 2025
- 2025VIDEOP2R: Video Understanding from Perception to Reasoning14 Nov 2025
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models6 Oct 2025
- 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Survey14 Aug 2025
- 2025AVC-DPO: Aligned Video Captioning via Direct Preference Optimization2 Jul 2025
Similar, not cited
Close in wording, no citation either way
- 2025ReasVQA: Advancing VideoQA with Imperfect Reasoning Process
- 2024Question-Instructed Visual Descriptions for Zero-Shot Video Question Answering
- 2022From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering
- 2025Team of One: Cracking Complex Video QA with Model Synergy
- 2024Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
- 2026Video-ToC: Video Tree-of-Cue Reasoning
- 2026UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
- 2025VideoMultiAgents: A Multi-Agent Framework for Video Question Answering