mlsanityData to 7 Oct 2026

VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocation

arXiv 2605.16079 ↗First posted 15 May 2026cs.CV cs.AI cs.HCcited by 1

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

also in 1 more
Abstract (arXiv)

Large Vision-Language Models (LVLMs) have shown significant progress in video understanding, yet they face substantial challenges in tasks requiring precise spatiotemporal localization at the instance level. Existing methods primarily rely on text prompts for human-model interaction, but these prompts struggle to provide precise spatial and temporal references, resulting in poor user experience. Furthermore, current approaches typically decouple visual perception from language reasoning, centering reasoning around language rather than visual content, which limits the model's ability to proactively perceive fine-grained visual evidence. To address these challenges, we propose VideoSeeker, a novel paradigm for instance-level video understanding through visual prompts. VideoSeeker seamlessly integrates agentic reasoning with instance-level video understanding tasks, enabling the model to proactively perceive and retrieve relevant video segments on demand. We construct a four-stage fully automated data synthesis pipeline to efficiently generate large-scale, high-quality instance-level video data. We internalize tool-calling and proactive perception capabilities into the model via cold-start supervision and RL training, building a powerful video understanding model. Experiments demonstrate that our model achieves an average improvement of +13.7% over baselines on instance-level video understanding tasks, surpassing powerful closed-source models such as GPT-4o and Gemini-2.5-Pro, while also showing effective transferability on general video understanding benchmarks. The relevant datasets and code will be released publicly.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 48 of its 53 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

48 of 53 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding15 Jan
  2. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling25 Nov
  3. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning27 Oct
  4. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning6 Aug
  5. 2025Scaling RL to Long Videos10 Jul
  6. 2025VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoning10 Apr
  7. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  8. 2025V2P-Bench: Evaluating Video-Language Understanding with Visual Prompts for Better Human-Model Interaction22 Mar
  9. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding17 Mar
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  11. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  12. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec

Other references · 36

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2025Qwen3-VL Technical Reportcited by 2,534
  7. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  8. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  9. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  10. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  11. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  12. 2025SAM 3: Segment Anything with Conceptscited by 835
24 more
  1. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  2. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 550
  3. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  4. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  5. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  6. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  7. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  8. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  9. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  10. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
  11. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 267
  12. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  13. 2025MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attentioncited by 248
  14. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 130
  15. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
  16. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
  17. 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learningcited by 116
  18. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuningcited by 86
  19. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 71
  20. 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 67
  21. 2025CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learningcited by 36
  22. 2026GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agentscited by 30
  23. 2025AdaTooler-V: Adaptive Tool-Use for Images and Videoscited by 20
  24. 2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Modelscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026LensWalk: Agentic Video Understanding by Planning How You See in Videos
  2. 2026VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking
  3. 2025Agentic Video Intelligence: A Flexible Framework for Advanced Video Exploration and Understanding
  4. 2026VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning
  5. 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaboration
  6. 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks
  7. 2026Agentic Video Understanding: A Survey
  8. 2026Act2See: Emergent Active Visual Perception for Video Reasoning