mlsanityData to 14 Oct 2026

LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling

arXiv 2511.20785 ↗First posted 25 Nov 2025cs.CVcited by 61

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Large multimodal models (LMMs) have shown great potential for video reasoning with textual Chain-of-Thought. However, they remain vulnerable to hallucinations, especially when processing long-form videos where evidence is sparse and temporally dispersed. Inspired by how humans comprehend long videos - by first skimming globally and then examining relevant clips for details - we introduce LongVT, an end-to-end agentic framework that enables "Thinking with Long Videos" via interleaved Multimodal Chain-of-Tool-Thought. Specifically, we exploit LMMs' inherent temporal grounding ability as a native video cropping tool to zoom in on a specific video clip and resample finer-grained video frames. This global-to-local reasoning loop continues until answers are grounded in retrieved visual evidence. Given the scarcity of fine-grained question-answering (QA) data for the long video reasoning task, we curate and will release a data suite named VideoSIAH to facilitate both training and evaluation. Specifically, our training dataset consists of 247.9K samples for tool-integrated cold-start supervised fine-tuning, 1.6K samples for agentic reinforcement learning, and 15.4K samples for agentic reinforcement fine-tuning, respectively. Our evaluation benchmark consists of 1,280 QA pairs that are carefully curated through a semi-automatic data pipeline with human-in-the-loop validation. With a meticulously designed three-stage training strategy and extensive empirical validation, LongVT consistently outperforms existing strong baselines across four challenging long-video understanding and reasoning benchmarks. Our codes, data, and model checkpoints are publicly available at https://github.com/EvolvingLMMs-Lab/LongVT .

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 56 of its 71 references that are in the map. First posted on arXiv 25 Nov 2025.

Builds on

56 of 71 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe20 Nov
  2. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning27 Oct
  3. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources25 Sep
  4. 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoning14 Aug
  5. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning6 Aug
  6. 2025Scaling RL to Long Videos10 Jul
  7. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
  8. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models24 May
  9. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
  10. 2025GRIT: Teaching MLLMs to Think with Images21 May
  11. 2025VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning18 May
  12. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
8 more
  1. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning9 Apr
  2. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  3. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles21 Mar
  4. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning10 Mar
  5. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  6. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  7. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
  8. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov

Other references · 36

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  10. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  11. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  12. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
24 more
  1. 2017Dense-Captioning Events in Videoscited by 793
  2. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  3. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  4. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  5. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  6. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  7. 2024LVBench: An Extreme Long Video Understanding Benchmarkcited by 410
  8. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
  9. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 363
  10. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  11. 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
  12. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 146
  13. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136
  14. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 97
  15. 2020Multimodal Pretraining for Dense Video Captioningcited by 79
  16. 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 68
  17. 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answeringcited by 65
  18. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 41
  19. 2023VidChapters-7M: Video Chapters at Scalecited by 39
  20. 2025SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learningcited by 34
  21. 2024Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignmentcited by 32
  22. 2025Improving LLM Video Understanding with 16 Frames Per Secondcited by 26
  23. 2025Multi-Agent Tool-Integrated Policy Optimizationcited by 16
  24. 2025TimeExpert: An Expert-Guided Video LLM for Video Temporal Groundingcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 56.

61 papers.

Sort:

By area:

  1. 2025Accelerating Streaming Video Large Language Models via Hierarchical Token Compressioncited by 37
  2. 2026Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoningcited by 24
  3. 2026PyVision-RL: Forging Open Agentic Vision Models via RLcited by 16
  4. 2026Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoningcited by 14
  5. 2026EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Usecited by 12
  6. 2026VISD: Enhancing Video Reasoning via Structured Self-Distillationcited by 10
  7. 2026Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baselinecited by 8
  8. 2026ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoningcited by 8
  9. 2026Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RLcited by 8
  10. 2026VideoTemp-o3: Harmonizing Temporal Grounding and Video Understanding in Agentic Thinking-with-Videoscited by 7
  11. 2026VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verificationcited by 7
  12. 2025SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learningcited by 6
38 more
  1. 2026VideoSEAL: Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authoritycited by 6
  2. 2026ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learningcited by 6
  3. 2025InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaborationcited by 5
  4. 2025A Benchmark for Omni-Modal Reasoning in Long Videoscited by 5
  5. 2026VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learningcited by 5
  6. 2026Where to Look Matters: On-Policy Self-Distillation for Long-Video Understandingcited by 5
  7. 2026VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Promptingcited by 4
  8. 2026StreamingClaw Technical Reportcited by 4
  9. 2026SVAgent: Storyline-Guided Long Video Understanding via Cross-Modal Multi-Agent Collaborationcited by 4
  10. 2026Bridging Modalities, Spanning Time: Structured Memory for Ultra-Long Agentic Video Reasoningcited by 4
  11. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4
  12. 2026Native Active Perception as Reasoning for Omni-Modal Understandingcited by 4
  13. 2026MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documentscited by 3
  14. 2026VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoningcited by 3
  15. 2026VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocationcited by 3
  16. 2026OmniReasoner: Thinking with Long Audio-Video via Native Tool Usecited by 3
  17. 2025Script: Graph-Structured and Query-Conditioned Semantic Token Pruning for Multimodal Large Language Modelscited by 2
  18. 2026MedScope: Incentivizing "Think with Videos" for Clinical Reasoning via Coarse-to-Fine Tool Callingcited by 2
  19. 2026Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Modelscited by 2
  20. 2026V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Modelscited by 2
  21. 2026See More, Think Deeper: Query-Expanded Visual Evidence and Answer-Clue Guided Reflection for Long Video Understandingcited by 2
  22. 2026Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoningcited by 2
  23. 2026Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understandingcited by 2
  24. 2026GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experiencecited by 2
  25. 2026StreamOPD: A Post-Training Recipe with Spatio-Temporal Cue Gating for Streaming Video Understandingcited by 2
  26. 2026Routing Before Looking: Query-Adaptive Evidence Acquisition for Long-form Video Understandingcited by 2
  27. 2026Online Video Agent Harness for Long Video Understandingcited by 2
  28. 2026ForeSea: AI Forensic Search with Multi-modal Queries for Video Surveillancecited by 1
  29. 2026Towards Temporal Compositional Reasoning in Long-Form Sports Videoscited by 1
  30. 2026Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Contextcited by 1
  31. 2026STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Modelscited by 1
  32. 2026DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understandingcited by 1
  33. 2026SVI-Bench: A Dynamic Microworld for Strategic Video Intelligencecited by 1
  34. 2026SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Seriescited by 1
  35. 2026MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMscited by 1
  36. 2026Thinking in Video: Can Video Generators Really Reason About the Real World?cited by 1
  37. 2026Thinking with Cameras: Active Visual Reasoning via Dynamic Viewpoint Control for Surveillance Video Understandingcited by 1
  38. 2026Visual Para-Thinker: Divide-and-Conquer Reasoning for Visual Comprehensioncited by 0

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Temporal Tree of Thought: Reasoning-Guided Visual Cue Search for Long-Video Understanding
  2. 2026Think with Grounding: Curriculum Reinforced Reasoning with Video Grounding for Long Video Understanding
  3. 2026Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervision
  4. 2026Semantic and Visual Evidence for Efficient Long-Video Reasoning: A Solution for the HD-EPIC VQA Challenge
  5. 2025Video Finetuning Improves Reasoning Between Frames
  6. 2026Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs
  7. 2025Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning
  8. 2025Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames