mlsanityData to 7 Oct 2026

VTimeLLM: Empower LLM to Grasp Video Moments

arXiv 2311.18445 ↗First posted 30 Nov 2023cs.CVcited by 253

Not on the map. Why?

Cited by 87 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have shown remarkable text understanding capabilities, which have been extended as Video LLMs to handle video data for comprehending visual details. However, existing Video LLMs can only provide a coarse description of the entire video, failing to capture the precise start and end time boundary of specific events. In this paper, we solve this issue via proposing VTimeLLM, a novel Video LLM designed for fine-grained video moment understanding and reasoning with respect to time boundary. Specifically, our VTimeLLM adopts a boundary-aware three-stage training strategy, which respectively utilizes image-text pairs for feature alignment, multiple-event videos to increase temporal-boundary awareness, and high-quality video-instruction tuning to further improve temporal understanding ability as well as align with human intents. Extensive experiments demonstrate that in fine-grained time-related comprehension tasks for videos such as Temporal Video Grounding and Dense Video Captioning, VTimeLLM significantly outperforms existing Video LLMs. Besides, benefits from the fine-grained temporal understanding of the videos further enable VTimeLLM to beat existing Video LLMs in video dialogue benchmark, showing its superior cross-modal understanding and reasoning abilities.

Key facts. Cited by 253 papers in the map (data through 7 Oct 2026). Builds on 32 of its 37 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

32 of 37 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  7. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  8. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  9. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  10. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  11. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  12. 2023VideoChat: Chat-Centric Video Understandingcited by 851
20 more
  1. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  2. 2017Dense-Captioning Events in Videoscited by 789
  3. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  4. 2017Localizing Moments in Video with Natural Languagecited by 660
  5. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  6. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  7. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
  8. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  9. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
  10. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 231
  11. 2018End-to-End Dense Video Captioning with Masked Transformercited by 214
  12. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  13. 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
  14. 2019Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videoscited by 135
  15. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  16. 2020A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformercited by 75
  17. 2018Bidirectional Attentive Fusion with Context Gating for Dense Video Captioningcited by 68
  18. 2018Weakly Supervised Dense Event Captioning in Videoscited by 58
  19. 2023BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuningcited by 41
  20. 2022A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approachcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 57; 2025: 114; 2026: 81.

253 papers.

Sort:

By area:

  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  2. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  3. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  4. 2023Video Understanding with Large Language Models: A Surveycited by 203
  5. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  6. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
  7. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  8. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 127
  9. 2024ST-LLM: Large Language Models Are Effective Temporal Learnerscited by 126
  10. 2024LITA: Language Instructed Temporal-Localization Assistantcited by 124
  11. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  12. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
38 more
  1. 2024VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understandingcited by 113
  2. 2024HourVideo: 1-Hour Video-Language Understandingcited by 110
  3. 2024GroundingGPT:Language Enhanced Multi-modal Grounding Modelcited by 101
  4. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 99
  5. 2024ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Modelscited by 89
  6. 2024Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Modelscited by 87
  7. 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 87
  8. 2024HawkEye: Training Video-Text LLMs for Grounding Text in Videoscited by 83
  9. 2024VoCo-LLaMA: Towards Vision Compression with Large Language Modelscited by 78
  10. 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
  11. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Abilitycited by 69
  12. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 68
  13. 2025V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoningcited by 66
  14. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  15. 2025VideoRoPE: What Makes for Good Video Rotary Position Embedding?cited by 63
  16. 2024Online Video Understanding: OVBench and VideoChat-Onlinecited by 58
  17. 2025LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understandingcited by 57
  18. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOcited by 57
  19. 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 56
  20. 2024ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Modelscited by 54
  21. 2024CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarioscited by 53
  22. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 52
  23. 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videoscited by 51
  24. 2024LongVALE: Vision-Audio-Language-Event Benchmark Towards Time-Aware Omni-Modal Perception of Long Videoscited by 49
  25. 2025Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understandingcited by 49
  26. 2024Elysium: Exploring Object-level Perception in Videos via MLLMcited by 48
  27. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 45
  28. 2024VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Formatcited by 44
  29. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 41
  30. 2024Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedbackcited by 36
  31. 2025VideoAgent2: Enhancing the LLM-Based Agent System for Long-Form Video Understanding by Uncertainty-Aware CoTcited by 36
  32. 2025VideoChat-A1: Thinking with Long Videos by Chain-of-Shot Reasoningcited by 32
  33. 2026Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillationcited by 32
  34. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 31
  35. 2024Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignmentcited by 29
  36. 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Modelscited by 28
  37. 2025SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capabilitycited by 28
  38. 2025Universal Video Temporal Grounding with Generative Multi-modal Large Language Modelscited by 26

50 most cited of 253 All 253 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025How Important are Videos for Training Video LLMs?
  2. 2023VideoLLM: Modeling Video Sequence with Large Language Models
  3. 2025RTime-QA: A Benchmark for Atomic Temporal Event Understanding in Large Multi-modal Models
  4. 2024TemporalVLM: Video LLMs for Temporal Reasoning in Long Videos
  5. 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understanding
  6. 2026ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting
  7. 2025SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding
  8. 2024LongVLM: Efficient Long Video Understanding via Large Language Models