mlsanityData to 7 Oct 2026

Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning

arXiv 2402.11435 ↗First posted 18 Feb 2024cs.CVcited by 122

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Large Language Models (LLMs) demonstrate remarkable proficiency in comprehending and handling text-based tasks. Many efforts are being made to transfer these attributes to video modality, which are termed Video-LLMs. However, existing Video-LLMs can only capture the coarse-grained semantics and are unable to effectively handle tasks related to comprehension or localization of specific video segments. In light of these challenges, we propose Momentor, a Video-LLM capable of accomplishing fine-grained temporal understanding tasks. To support the training of Momentor, we design an automatic data generation engine to construct Moment-10M, a large-scale video instruction dataset with segment-level instruction data. We train Momentor on Moment-10M, enabling it to perform segment-level reasoning and localization. Zero-shot evaluations on several tasks demonstrate that Momentor excels in fine-grained temporally grounded comprehension and localization.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 58 references that are in the map. First posted on arXiv 18 Feb 2024.

Builds on

34 of 58 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2017Mask R-CNNcited by 7,259
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  11. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  12. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
22 more
  1. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  2. 2017Dense-Captioning Events in Videoscited by 789
  3. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  4. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  5. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  6. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  7. 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
  8. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 283
  9. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  10. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 253
  11. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
  12. 2023Valley: Video Assistant with Large Language model Enhanced abilitYcited by 231
  13. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 189
  14. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  15. 2015Sequence to Sequence -- Video to Textcited by 157
  16. 2020TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Taskscited by 77
  17. 2022Fine-Grained Semantically Aligned Vision-Language Pre-Trainingcited by 68
  18. 2022Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learningcited by 59
  19. 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
  20. 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
  21. 2023Panoptic Video Scene Graph Generationcited by 46
  22. 2023Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Groundingcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 25; 2025: 56; 2026: 41.

122 papers.

Sort:

By area:

  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  2. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  3. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  4. 2024LITA: Language Instructed Temporal-Localization Assistantcited by 124
  5. 2024Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Modelscited by 87
  6. 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 87
  7. 2025BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understandingcited by 72
  8. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Abilitycited by 69
  9. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 68
  10. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  11. 2025LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understandingcited by 57
  12. 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videoscited by 51
38 more
  1. 2024Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editingcited by 49
  2. 2025TLA: Tactile-Language-Action Model for Contact-Rich Manipulationcited by 46
  3. 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videoscited by 43
  4. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 41
  5. 2024From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understandingcited by 29
  6. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Groundingcited by 28
  7. 2025Universal Video Temporal Grounding with Generative Multi-modal Large Language Modelscited by 26
  8. 2025SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understandingcited by 25
  9. 2025Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiencycited by 25
  10. 2024STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Trainingcited by 22
  11. 2025A Survey of Reasoning and Agentic Systems in Time Series with Large Language Modelscited by 21
  12. 2024VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridgescited by 20
  13. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 19
  14. 2024On the Consistency of Video Large Language Models in Temporal Comprehensioncited by 18
  15. 2024ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videoscited by 18
  16. 2025Datasets and Recipes for Video Temporal Grounding via Reinforcement Learningcited by 18
  17. 2024Chrono: A Simple Blueprint for Representing Time in MLLMscited by 17
  18. 2025Time Blindness: Why Video-Language Models Can't See What Humans Can?cited by 17
  19. 2024TimeRefine: Temporal Grounding with Time Refining Video LLMcited by 15
  20. 2025TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMscited by 15
  21. 2024Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridgecited by 14
  22. 2025VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Modelscited by 14
  23. 2025TimeExpert: An Expert-Guided Video LLM for Video Temporal Groundingcited by 14
  24. 2025Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generationcited by 13
  25. 2025TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Actioncited by 13
  26. 2024Seq2Time: Sequential Knowledge Transfer for Video LLM Temporal Groundingcited by 12
  27. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMscited by 12
  28. 2024TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignmentcited by 10
  29. 2024Mastering Collaborative Multi-modal Data Selection: A Focus on Informativeness, Uniqueness, and Representativenesscited by 10
  30. 2025iMOVE: Instance-Motion-Aware Video Understandingcited by 10
  31. 2025ZoomV: Temporal Zoom-in for Efficient Long Video Understandingcited by 10
  32. 2025Benchmarking Multimodal CoT Reward Model Stepwise by Visual Programcited by 10
  33. 2025Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenderscited by 10
  34. 2025VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioningcited by 9
  35. 2024TemporalVLM: Video LLMs for Temporal Reasoning in Long Videoscited by 8
  36. 2025When Continue Learning Meets Multimodal Large Language Model: A Surveycited by 8
  37. 2025VideoExpert: Augmented LLM for Temporal-Sensitive Video Understandingcited by 8
  38. 2026CoPE-VideoLM: Leveraging Codec Primitives For Efficient Video Language Modelingcited by 8

50 most cited of 122 All 122 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Do Language Models Understand Time?
  2. 2024Temporal Reasoning Transfer from Text to Video
  3. 2025How Important are Videos for Training Video LLMs?
  4. 2025Enhancing Temporal Understanding in Video-LLMs through Stacked Temporal Attention in Vision Encoders
  5. 2026Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervision
  6. 2026Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
  7. 2024VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs
  8. 2023VideoLLM: Modeling Video Sequence with Large Language Models