mlsanityData to 14 Oct 2026

A Survey on Video Temporal Grounding with Multimodal Large Language Model

arXiv 2508.10922 ↗First posted 7 Aug 2025cs.CVcited by 20

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

The recent advancement in video temporal grounding (VTG) has significantly enhanced fine-grained video understanding, primarily driven by multimodal large language models (MLLMs). With superior multimodal comprehension and reasoning abilities, VTG approaches based on MLLMs (VTG-MLLMs) are gradually surpassing traditional fine-tuned methods. They not only achieve competitive performance but also excel in generalization across zero-shot, multi-task, and multi-domain settings. Despite extensive surveys on general video-language understanding, comprehensive reviews specifically addressing VTG-MLLMs remain scarce. To fill this gap, this survey systematically examines current research on VTG-MLLMs through a three-dimensional taxonomy: 1) the functional roles of MLLMs, highlighting their architectural significance; 2) training paradigms, analyzing strategies for temporal reasoning and task adaptation; and 3) video feature processing techniques, which determine spatiotemporal representation effectiveness. We further discuss benchmark datasets, evaluation protocols, and summarize empirical findings. Finally, we identify existing limitations and propose promising research directions. For additional resources and details, readers are encouraged to visit our repository at https://github.com/ki-lw/Awesome-MLLMs-for-Video-Temporal-Grounding.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 145 of its 180 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

145 of 180 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding27 May
  2. 2025Slow-Fast Architecture for Video Multi-Modal Large Language Models2 Apr
  3. 2025SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capability18 Mar
  4. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding17 Mar
  5. 2025AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding16 Mar
  6. 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding11 Mar
  7. 2025LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding14 Jan
  8. 2025Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models14 Jan
  9. 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding16 Dec
  10. 2024VideoLights: Feature Refinement and Cross-Task Alignment Transformer for Joint Video Highlight Detection and Moment Retrieval2 Dec
  11. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability27 Nov
  12. 2024ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos22 Nov
11 more
  1. 2024LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrieval21 Nov
  2. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning25 Oct
  3. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models9 Aug
  4. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning25 Apr
  5. 2024LITA: Language Instructed Temporal-Localization Assistant27 Mar
  6. 2024Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge25 Feb
  7. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
  8. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
  10. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun
  11. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning27 Feb

Other references · 122

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  8. 2023Visual Instruction Tuningcited by 7,869
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
38 more
  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  7. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  8. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  9. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  10. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  11. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  12. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  13. 2023VideoChat: Chat-Centric Video Understandingcited by 857
  14. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  15. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  16. 2017Dense-Captioning Events in Videoscited by 793
  17. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  18. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
  19. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  20. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  21. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  22. 2017Localizing Moments in Video with Natural Languagecited by 663
  23. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  24. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
  25. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  26. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  27. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  28. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 537
  29. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
  30. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  31. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  32. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  33. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 388
  34. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  35. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  36. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  37. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  38. 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255

50 most cited of 122 All 122 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 13.

20 papers.

Sort:

By area:

  1. 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 28
  2. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 24
  3. 2025Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoningcited by 14
  4. 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
  5. 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioningcited by 8
  6. 2025ChronusOmni: Improving Time Awareness of Omni Large Language Modelscited by 6
  7. 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMscited by 4
  8. 2025FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensingcited by 3
  9. 2025When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactionscited by 3
  10. 2026Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Groundingcited by 3
  11. 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigmscited by 3
  12. 2026LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streamscited by 2
8 more
  1. 2026Pixelis: Reasoning in Pixels, from Seeing to Actingcited by 1
  2. 2026The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentationcited by 1
  3. 2026MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cuescited by 1
  4. 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
  5. 2026Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillationcited by 1
  6. 2026Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompt: The 1st Winner for 5th PVUW MOSE Challengecited by 0
  7. 2026Static and Dynamic Graph Alignment Network for Temporal Video Groundingcited by 0
  8. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs
  2. 2026GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Grounding
  3. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding
  4. 2025Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Grounding
  5. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMs
  6. 2026OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding
  7. 2026UniversalVTG: A Universal and Lightweight Foundation Model for Video Temporal Grounding
  8. 2025ExpVG: Investigating the Design Space of Visual Grounding in Multimodal Large Language Model