mlsanityData to 14 Oct 2026

Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding

arXiv 2503.13377 ↗First posted 17 Mar 2025cs.CV cs.AI cs.CLcited by 146

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Temporal Video Grounding (TVG), the task of locating specific video segments based on language queries, is a core challenge in long-form video understanding. While recent Large Vision-Language Models (LVLMs) have shown early promise in tackling TVG through supervised fine-tuning (SFT), their abilities to generalize remain limited. To address this, we propose a novel post-training framework that enhances the generalization capabilities of LVLMs via reinforcement learning (RL). Specifically, our contributions span three key directions: (1) Time-R1: we introduce a reasoning-guided post-training framework via RL with verifiable reward to enhance the capabilities of LVLMs on the TVG task. (2) TimeRFT: we explore data-efficient post-training strategies on our curated RL-friendly dataset, which trains the model to progressively comprehend difficult samples, leading to better generalization. (3) TVGBench: we carefully construct a small yet comprehensive benchmark for LVLM evaluation, assessing 11 types of queries and featuring balanced distributions across both videos and queries. Extensive experiments demonstrate that Time-R1 achieves state-of-the-art performance across multiple downstream datasets using only 2.5K training data, while improving its general video understanding capabilities.

Key facts. Cited by 146 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 48 of its 71 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

48 of 71 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling31 Dec
  2. 2024ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos22 Nov
  3. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning25 Oct
  4. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  5. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
  6. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov
  7. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning27 Feb

Other references · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  10. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  11. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  12. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
29 more
  1. 2017Localizing Moments in Video with Natural Languagecited by 663
  2. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  3. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  4. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 537
  5. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  6. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  7. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  8. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 363
  9. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  10. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
  11. 2022Egocentric Video-Language Pretrainingcited by 232
  12. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 190
  13. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 138
  14. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  15. 2025EgoLife: Towards Egocentric Life Assistantcited by 91
  16. 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 89
  17. 2024HawkEye: Training Video-Text LLMs for Grounding Text in Videoscited by 86
  18. 2024VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Groundingcited by 85
  19. 2023Hierarchical Video-Moment Retrieval and Step-Captioningcited by 83
  20. 2023Knowing Where to Focus: Event-aware Transformer for Video Groundingcited by 79
  21. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  22. 2024ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Modelscited by 54
  23. 2021A Closer Look at Temporal Sentence Grounding in Videos: Dataset and Metriccited by 42
  24. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  25. 2024SnAG: Scalable and Accurate Video Groundingcited by 36
  26. 2020Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 29
  27. 2024Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignmentcited by 29
  28. 2023Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Groundingcited by 19
  29. 2025iMOVE: Instance-Motion-Aware Video Understandingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 51; 2026: 95.

146 papers.

Sort:

By area:

  1. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 91
  2. 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 91
  3. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  4. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 61
  5. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
  6. 2025WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoningcited by 53
  7. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 50
  8. 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidencecited by 49
  9. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 39
  10. 2026Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillationcited by 36
  11. 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learningcited by 34
  12. 2025Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoningcited by 33
38 more
  1. 2025VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinkingcited by 30
  2. 2025AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestrationcited by 27
  3. 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 26
  4. 2025Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thoughtcited by 24
  5. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 24
  6. 2025UI-Ins: Enhancing GUI Grounding with Multi-Perspective Instruction-as-Reasoningcited by 22
  7. 2025AdaTooler-V: Adaptive Tool-Use for Images and Videoscited by 21
  8. 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twicecited by 21
  9. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 20
  10. 2025Datasets and Recipes for Video Temporal Grounding via Reinforcement Learningcited by 19
  11. 2025Shuffle-R1: Efficient RL framework for Multimodal Large Language Models via Data-centric Dynamic Shufflecited by 19
  12. 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 16
  13. 2025Reinforcing Video Reasoning Segmentation to Think Before It Segmentscited by 15
  14. 2025Xiaomi MiMo-VL-Miloco Technical Reportcited by 15
  15. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14
  16. 2025SiLVR: A Simple Language-based Video Reasoning Frameworkcited by 14
  17. 2025Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning without 3D Annotationscited by 14
  18. 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
  19. 2025Thinking With Bounding Boxes: Enhancing Spatio-Temporal Video Grounding via Reinforcement Fine-Tuningcited by 12
  20. 2025Improving the Reasoning of Multi-Image Grounding in MLLMs via Reinforcement Learningcited by 11
  21. 2026TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captionscited by 11
  22. 2025ZoomV: Temporal Zoom-in for Efficient Long Video Understandingcited by 10
  23. 2026VISD: Enhancing Video Reasoning via Structured Self-Distillationcited by 10
  24. 2025Prefix Grouper: Efficient GRPO Training through Shared-Prefix Forwardcited by 9
  25. 2025Tempo-R0: A Video-MLLM for Temporal Video Grounding through Efficient Temporal Sensing Reinforcement Learningcited by 9
  26. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  27. 2026CamReasoner: Reinforcing Camera Movement Understanding via Structured Spatial Reasoningcited by 9
  28. 2025MOSS-ChatV: Reinforcement Learning with Process Reasoning Reward for Video Temporal Reasoningcited by 8
  29. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
  30. 2025VIDEOP2R: Video Understanding from Perception to Reasoningcited by 8
  31. 2025TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learningcited by 8
  32. 2025Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-incited by 8
  33. 2026ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoningcited by 8
  34. 2025VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotationscited by 7
  35. 2026STReasoner: Empowering LLMs for Spatio-Temporal Reasoning in Time Series via Spatial-Aware Reinforcement Learningcited by 7
  36. 2026ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decodingcited by 7
  37. 2025Video-STR: Reinforcing MLLMs in Video Spatio-Temporal Reasoning with Relation Graphcited by 6
  38. 2025IC-World: In-Context Generation for Shared World Modelingcited by 6

50 most cited of 146 All 146 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024AutoTVG: A New Vision-language Pre-training Paradigm for Temporal Video Grounding
  2. 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Models
  3. 2025Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding
  4. 2025MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding
  5. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding
  6. 2024Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models
  7. 2026GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Grounding
  8. 2026UniversalVTG: A Universal and Lightweight Foundation Model for Video Temporal Grounding