VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocation
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
also in 1 more
Abstract (arXiv)
Large Vision-Language Models (LVLMs) have shown significant progress in video understanding, yet they face substantial challenges in tasks requiring precise spatiotemporal localization at the instance level. Existing methods primarily rely on text prompts for human-model interaction, but these prompts struggle to provide precise spatial and temporal references, resulting in poor user experience. Furthermore, current approaches typically decouple visual perception from language reasoning, centering reasoning around language rather than visual content, which limits the model's ability to proactively perceive fine-grained visual evidence. To address these challenges, we propose VideoSeeker, a novel paradigm for instance-level video understanding through visual prompts. VideoSeeker seamlessly integrates agentic reasoning with instance-level video understanding tasks, enabling the model to proactively perceive and retrieve relevant video segments on demand. We construct a four-stage fully automated data synthesis pipeline to efficiently generate large-scale, high-quality instance-level video data. We internalize tool-calling and proactive perception capabilities into the model via cold-start supervision and RL training, building a powerful video understanding model. Experiments demonstrate that our model achieves an average improvement of +13.7% over baselines on instance-level video understanding tasks, surpassing powerful closed-source models such as GPT-4o and Gemini-2.5-Pro, while also showing effective transferability on general video understanding benchmarks. The relevant datasets and code will be released publicly.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 48 of its 53 references that are in the map. First posted on arXiv 15 May 2026.
Builds on
48 of 53 references.
Closest ancestors · 12
Same direction, newest first
- 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding15 Jan
- 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling25 Nov
- 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning27 Oct
- 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning6 Aug
- 2025Scaling RL to Long Videos10 Jul
- 2025VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoning10 Apr
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2025V2P-Bench: Evaluating Video-Language Understanding with Visual Prompts for Better Human-Model Interaction22 Mar
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding17 Mar
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
Other references · 36
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024GPT-4o System Cardcited by 4,467
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
- 2025SAM 3: Segment Anything with Conceptscited by 835
24 more
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
- 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
- 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 267
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
- 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
- 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 130
- 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
- 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
- 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learningcited by 116
- 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 86
- 2025PyVision: Agentic Vision with Dynamic Toolingcited by 71
- 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 67
- 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningcited by 36
- 2026GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agentscited by 30
- 2025AdaTooler-V: Adaptive Tool-Use for Images and Videoscited by 20
- 2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Modelscited by 11
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026LensWalk: Agentic Video Understanding by Planning How You See in Videos
- 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking
- 2025Agentic Video Intelligence: A Flexible Framework for Advanced Video Exploration and Understanding
- 2026VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning
- 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaboration
- 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks
- 2026Agentic Video Understanding: A Survey
- 2026Act2See: Emergent Active Visual Perception for Video Reasoning