# ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding

arXiv: https://arxiv.org/abs/2506.01300 · HTML: https://mlsanity.com/paper/2506.01300/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 49 of its 66 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)
- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 30 of 49)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021)
- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [ShareGPT4Video: Improving Video Understanding and Generation with Better Captions](https://mlsanity.com/paper/2406.04325/) (2024)
- [TempCompass: Do Video LLMs Really Understand Videos?](https://mlsanity.com/paper/2403.00476/) (2024)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)
- [MERLOT: Multimodal Neural Script Knowledge Models](https://mlsanity.com/paper/2106.02636/) (2021)
- [Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning](https://mlsanity.com/paper/2302.14115/) (2023)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Inference-Time Scaling for Generalist Reward Modeling](https://mlsanity.com/paper/2504.02495/) (2025)

## Built on by (most cited first, 30 of 50)

- [DBLoss: Decomposition-based Loss Function for Time Series Forecasting](https://mlsanity.com/paper/2510.23672/) (2025)
- [Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspective](https://mlsanity.com/paper/2510.14510/) (2025)
- [StreamAgent: Towards Anticipatory Agents for Streaming Video Understanding](https://mlsanity.com/paper/2508.01875/) (2025)
- [Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning](https://mlsanity.com/paper/2511.19900/) (2025)
- [Aurora: Towards Universal Generative Multimodal Time Series Forecasting](https://mlsanity.com/paper/2509.22295/) (2025)
- [Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding](https://mlsanity.com/paper/2512.05774/) (2025)
- [TimeART: Towards Agentic Time Series Reasoning via Tool-Augmentation](https://mlsanity.com/paper/2601.13653/) (2026)
- [LensWalk: Agentic Video Understanding by Planning How You See in Videos](https://mlsanity.com/paper/2603.24558/) (2026)
- [PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents](https://mlsanity.com/paper/2510.00413/) (2025)
- [Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understanding](https://mlsanity.com/paper/2603.17307/) (2026)
- [ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecasting](https://mlsanity.com/paper/2509.23313/) (2025)
- [Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models](https://mlsanity.com/paper/2510.05034/) (2025)
- [NavigScene: Bridging Local Perception and Global Navigation for Beyond-Visual-Range Autonomous Driving](https://mlsanity.com/paper/2507.05227/) (2025)
- [Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails](https://mlsanity.com/paper/2510.04860/) (2025)
- [STAR: Boosting Time Series Foundation Models for Anomaly Detection through State-aware Adapter](https://mlsanity.com/paper/2510.16014/) (2025)
- [Agentic Video Intelligence: A Flexible Framework for Advanced Video Exploration and Understanding](https://mlsanity.com/paper/2511.14446/) (2025)
- [MEXA: Towards General Multimodal Reasoning with Dynamic Multi-Expert Aggregation](https://mlsanity.com/paper/2506.17113/) (2025)
- [VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2511.19524/) (2025)
- [Learning an Efficient Optimizer via Hybrid-Policy Sub-Trajectory Balance](https://mlsanity.com/paper/2511.00543/) (2025)
- [Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs](https://mlsanity.com/paper/2512.06258/) (2025)
- [Adaptive Detector-Verifier Framework for Zero-Shot Polyp Detection in Open-World Settings](https://mlsanity.com/paper/2512.12492/) (2025)
- [Scaling Video Understanding via Compact Latent Multi-Agent Collaboration](https://mlsanity.com/paper/2605.00444/) (2026)
- [FLAME: Flow Enhanced Legendre Memory Models for General Time Series Forecasting](https://mlsanity.com/paper/2512.14253/) (2025)
- [TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents](https://mlsanity.com/paper/2605.09934/) (2026)
- [Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning](https://mlsanity.com/paper/2605.28192/) (2026)
- [Task-Aware Mixture-of-Experts for Time Series Analysis](https://mlsanity.com/paper/2509.22279/) (2025)
- [Are Video Reasoning Models Ready to Go Outside?](https://mlsanity.com/paper/2603.10652/) (2026)
- [GLANCE: A Global-Local Coordination Multi-Agent Framework for Music-Grounded Non-Linear Video Editing](https://mlsanity.com/paper/2604.05076/) (2026)
- [CollabSim: A CSCW-Grounded Methodology for Investigating Collaborative Competence of LLM Agents through Controlled Multi-Agent Experiments](https://mlsanity.com/paper/2606.06399/) (2026)
- [Low-Cost Test-Time Adaptation for Robust Video Editing](https://mlsanity.com/paper/2507.21858/) (2025)

## Abstract (arXiv)

Video understanding is fundamental to tasks such as action recognition, video reasoning, and robotic control. Early video understanding methods based on large vision-language models (LVLMs) typically adopt a single-pass reasoning paradigm without dynamic feedback, limiting the model's capacity to self-correct and adapt in complex scenarios. Recent efforts have attempted to address this limitation by incorporating reward models and reinforcement learning to enhance reasoning, or by employing tool-agent frameworks. However, these approaches face several challenges, including high annotation costs, reward signals that fail to capture real-time reasoning states, and low inference efficiency. To overcome these issues, we propose ReAgent-V, a novel agentic video understanding framework that integrates efficient frame selection with real-time reward generation during inference. These reward signals not only guide iterative answer refinement through a multi-perspective reflection mechanism-adjusting predictions from conservative, neutral, and aggressive viewpoints-but also enable automatic filtering of high-quality data for supervised fine-tuning (SFT), direct preference optimization (DPO), and group relative policy optimization (GRPO). ReAgent-V is lightweight, modular, and extensible, supporting flexible tool integration tailored to diverse tasks. Extensive experiments on 12 datasets across three core applications-video understanding, video reasoning enhancement, and vision-language-action model alignment-demonstrate significant gains in generalization and reasoning, with improvements of up to 6.9%, 2.1%, and 9.8%, respectively, highlighting the effectiveness and versatility of the proposed framework.