# ViaRL: Adaptive Temporal Grounding via Visual Iterated Amplification Reinforcement Learning

arXiv: https://arxiv.org/abs/2505.15447 · HTML: https://mlsanity.com/paper/2505.15447/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 32 of its 37 references that are in the map.
- First posted on arXiv 21 May 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [LLaVA-Video: Video Instruction Tuning With Synthetic Data](https://mlsanity.com/paper/2410.02713/) (2024)
- [MovieChat: From Dense Token to Sparse Memory for Long Video Understanding](https://mlsanity.com/paper/2307.16449/) (2023)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding](https://mlsanity.com/paper/2312.02051/) (2023)
- [ShareGPT4Video: Improving Video Understanding and Generation with Better Captions](https://mlsanity.com/paper/2406.04325/) (2024)
- [Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding](https://mlsanity.com/paper/2409.14485/) (2024)
- [InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling](https://mlsanity.com/paper/2501.12386/) (2025)
- [ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models](https://mlsanity.com/paper/2310.10505/) (2023)
- [Adaptive Keyframe Sampling for Long Video Understanding](https://mlsanity.com/paper/2502.21271/) (2025)
- [Supervising strong learners by amplifying weak experts](https://mlsanity.com/paper/1810.08575/) (2018)
- [Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input](https://mlsanity.com/paper/2408.15542/) (2024)
- [Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videos](https://mlsanity.com/paper/2408.14023/) (2024)
- [M-LLM Based Video Frame Selection for Efficient Video Understanding](https://mlsanity.com/paper/2502.19680/) (2025)
- [Frame-Voyager: Learning to Query Frames for Video Large Language Models](https://mlsanity.com/paper/2410.03226/) (2024)
- [Number it: Temporal Grounding Videos like Flipping Manga](https://mlsanity.com/paper/2411.10332/) (2024)
- [KeyVideoLLM: Towards Large-scale Video Keyframe Selection](https://mlsanity.com/paper/2407.03104/) (2024)

## Built on by (most cited first, 11 of 11)

- [Phi-Ground Tech Report: Advancing Perception in GUI Grounding](https://mlsanity.com/paper/2507.23779/) (2025)
- [E-VRAG: Enhancing Long Video Understanding with Resource-Efficient Retrieval Augmented Generation](https://mlsanity.com/paper/2508.01546/) (2025)
- [K-frames: Scene-Driven Any-k Keyframe Selection for long video understanding](https://mlsanity.com/paper/2510.13891/) (2025)
- [From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understanding](https://mlsanity.com/paper/2510.02262/) (2025)
- [Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models](https://mlsanity.com/paper/2510.05034/) (2025)
- [FrameOracle: Learning What to See and How Much to See in Videos](https://mlsanity.com/paper/2510.03584/) (2025)
- [MSJoE: Jointly Evolving MLLM and Sampler for Efficient Long-Form Video Understanding](https://mlsanity.com/paper/2602.22932/) (2026)
- [FocusGraph: Graph-Structured Frame Selection for Embodied Long Video Question Answering](https://mlsanity.com/paper/2603.04349/) (2026)
- [Swift Sampling: Selecting Temporal Surprises via Taylor Series](https://mlsanity.com/paper/2605.22678/) (2026)
- [HORNet: Task-Guided Frame Selection for Video Question Answering with Vision-Language Models](https://mlsanity.com/paper/2603.18850/) (2026)
- [Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs](https://mlsanity.com/paper/2609.10355/) (2026)

## Abstract (arXiv)

Video understanding is inherently intention-driven-humans naturally focus on relevant frames based on their goals. Recent advancements in multimodal large language models (MLLMs) have enabled flexible query-driven reasoning; however, video-based frameworks like Video Chain-of-Thought lack direct training signals to effectively identify relevant frames. Current approaches often rely on heuristic methods or pseudo-label supervised annotations, which are both costly and limited in scalability across diverse scenarios. To overcome these challenges, we introduce ViaRL, the first framework to leverage rule-based reinforcement learning (RL) for optimizing frame selection in intention-driven video understanding. An iterated amplification strategy is adopted to perform alternating cyclic training in the video CoT system, where each component undergoes iterative cycles of refinement to improve its capabilities. ViaRL utilizes the answer accuracy of a downstream model as a reward signal to train a frame selector through trial-and-error, eliminating the need for expensive annotations while closely aligning with human-like learning processes. Comprehensive experiments across multiple benchmarks, including VideoMME, LVBench, and MLVU, demonstrate that ViaRL consistently delivers superior temporal grounding performance and robust generalization across diverse video understanding tasks, highlighting its effectiveness and scalability. Notably, ViaRL achieves a nearly 15% improvement on Needle QA, a subset of MLVU, which is required to search a specific needle within a long video and regarded as one of the most suitable benchmarks for evaluating temporal grounding.