mlsanityData to 14 Oct 2026

VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding

arXiv 2609.15606 ↗First posted 14 Sep 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have achieved remarkable progress on short video understanding yet remain limited on long videos due to the limited visual context window. Prevailing approaches rely on uniform frame sampling or recent coarse-to-fine agentic zooming, both of which struggle to localize sparse, decisive evidence in sufficiently long videos. We formulate long video understanding as a Sequential Evidence Acquisition (SEA) problem, in which an agent reads the video turn by turn along the temporal axis, deciding at each turn how fast to watch, what evidence to retain, when to revisit uncertain segments, and when to stop and answer. Inspired by this view, we propose VideoScout, a multi-turn reasoning agent that instantiates the SEA paradigm through adaptive reasoning pacing. Specifically, by dynamically controlling the viewing pace, VideoScout enables efficient traversal of long videos within a bounded visual context window, allowing the agent to access more video content while balancing content analysis depth with reading efficiency. To train VideoScout, we construct VideoScout-66K, a set of over 66K high-quality exploration turns from 10K answer-verified trajectories, and adopt a two-stage pipeline: cold-start supervised fine-tuning teaches the agent per-turn output format, while the Decoupled Clip and Dynamic sAmpling Policy Optimization (DAPO) algorithm performs trajectory-level reinforcement learning with a composite reward that jointly considers answer accuracy, output format compliance, and the temporal alignment between the agent's viewing progress and the teacher's answer timing measured by intersection-over-union (IoU). Extensive experiments on long video understanding and reasoning benchmarks demonstrate that our 7B model achieves strong performance compared with existing trained 7B agentic models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 87 of its 92 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

87 of 92 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Video-CoM: Interactive Video Reasoning via Chain of Manipulations28 Nov
  2. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling25 Nov
  3. 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence23 Oct
  4. 2025Video-MTR: Reinforced Multi-Turn Reasoning for Long Video Understanding28 Aug
  5. 2025Scaling RL to Long Videos10 Jul
  6. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning18 May
  7. 2025MINERVA: Evaluating Complex Video Reasoning1 May
  8. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning9 Apr
  9. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  10. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning17 Mar
  11. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  12. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
3 more
  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  2. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos29 May
  3. 2024MoReVQA: Exploring Modular Reasoning Models for Video Question Answering9 Apr

Other references · 72

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  10. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  11. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  12. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
38 more
  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  2. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  3. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  4. 2024Self-Rewarding Language Modelscited by 681
  5. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  6. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  7. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  8. 2024Long Context Transfer from Language to Visioncited by 515
  9. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  10. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  11. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
  12. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
  13. 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
  14. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
  15. 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
  16. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
  17. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 229
  18. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 213
  19. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  20. 2020MovieNet: A Holistic Dataset for Movie Understandingcited by 206
  21. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
  22. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 178
  23. 2025Adaptive Keyframe Sampling for Long Video Understandingcited by 167
  24. 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
  25. 2025MMVU: Measuring Expert-Level Multi-Discipline Video Understandingcited by 137
  26. 2024MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokenscited by 118
  27. 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 98
  28. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 83
  29. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
  30. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  31. 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 65
  32. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOcited by 61
  33. 2024VCA: Video Curious Agent for Long Video Understandingcited by 56
  34. 2024KeyVideoLLM: Towards Large-scale Video Keyframe Selectioncited by 54
  35. 2023Learning Procedure-aware Video Representation from Instructional Videos and Their Narrationscited by 46
  36. 2024Goldfish: Vision-Language Understanding of Arbitrarily Long Videoscited by 40
  37. 2021Reconstructive Sequence-Graph Network for Video Summarizationcited by 34
  38. 2023HowToCaption: Prompting LLMs to Transform Video Annotations at Scalecited by 34

50 most cited of 72 All 72 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Adaptive Video Understanding Agent: Enhancing efficiency with dynamic frame sampling and feedback-driven reasoning
  2. 2026LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding
  3. 2026VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning
  4. 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking
  5. 2026Temporal Tree of Thought: Reasoning-Guided Visual Cue Search for Long-Video Understanding
  6. 2025Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
  7. 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning
  8. 2026Agentic Video Understanding: A Survey