VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding
On the map: LLM reasoning → Multimodal reasoning
also in 2 more
Abstract (arXiv)
Multimodal Large Language Models (MLLMs) have achieved remarkable progress on short video understanding yet remain limited on long videos due to the limited visual context window. Prevailing approaches rely on uniform frame sampling or recent coarse-to-fine agentic zooming, both of which struggle to localize sparse, decisive evidence in sufficiently long videos. We formulate long video understanding as a Sequential Evidence Acquisition (SEA) problem, in which an agent reads the video turn by turn along the temporal axis, deciding at each turn how fast to watch, what evidence to retain, when to revisit uncertain segments, and when to stop and answer. Inspired by this view, we propose VideoScout, a multi-turn reasoning agent that instantiates the SEA paradigm through adaptive reasoning pacing. Specifically, by dynamically controlling the viewing pace, VideoScout enables efficient traversal of long videos within a bounded visual context window, allowing the agent to access more video content while balancing content analysis depth with reading efficiency. To train VideoScout, we construct VideoScout-66K, a set of over 66K high-quality exploration turns from 10K answer-verified trajectories, and adopt a two-stage pipeline: cold-start supervised fine-tuning teaches the agent per-turn output format, while the Decoupled Clip and Dynamic sAmpling Policy Optimization (DAPO) algorithm performs trajectory-level reinforcement learning with a composite reward that jointly considers answer accuracy, output format compliance, and the temporal alignment between the agent's viewing progress and the teacher's answer timing measured by intersection-over-union (IoU). Extensive experiments on long video understanding and reasoning benchmarks demonstrate that our 7B model achieves strong performance compared with existing trained 7B agentic models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 87 of its 92 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
87 of 92 references.
Closest ancestors · 15
Same direction, newest first
- 2025Video-CoM: Interactive Video Reasoning via Chain of Manipulations28 Nov
- 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling25 Nov
- 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence23 Oct
- 2025Video-MTR: Reinforced Multi-Turn Reasoning for Long Video Understanding28 Aug
- 2025Scaling RL to Long Videos10 Jul
- 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning18 May
- 2025MINERVA: Evaluating Complex Video Reasoning1 May
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning9 Apr
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning17 Mar
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
3 more
Other references · 72
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
38 more
- 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
- 2023VideoChat: Chat-Centric Video Understandingcited by 857
- 2024Self-Rewarding Language Modelscited by 681
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
- 2024Long Context Transfer from Language to Visioncited by 515
- 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
- 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
- 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
- 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
- 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
- 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
- 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 229
- 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 213
- 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
- 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
- 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
- 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 178
- 2025Adaptive Keyframe Sampling for Long Video Understandingcited by 167
- 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
- 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 137
- 2024MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokenscited by 118
- 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 98
- 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 83
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
- 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
- 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 65
- 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOcited by 61
- 2024VCA: Video Curious Agent for Long Video Understandingcited by 56
- 2024KeyVideoLLM: Towards Large-scale Video Keyframe Selectioncited by 54
- 2023Learning Procedure-aware Video Representation from Instructional Videos and Their Narrationscited by 46
- 2024Goldfish: Vision-Language Understanding of Arbitrarily Long Videoscited by 40
- 2021Reconstructive Sequence-Graph Network for Video Summarizationcited by 34
- 2023HowToCaption: Prompting LLMs to Transform Video Annotations at Scalecited by 34
50 most cited of 72 All 72 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Adaptive Video Understanding Agent: Enhancing efficiency with dynamic frame sampling and feedback-driven reasoning
- 2026LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding
- 2026VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning
- 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking
- 2026Temporal Tree of Thought: Reasoning-Guided Visual Cue Search for Long-Video Understanding
- 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
- 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning
- 2026Agentic Video Understanding: A Survey