A Survey on Video Temporal Grounding with Multimodal Large Language Model
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
The recent advancement in video temporal grounding (VTG) has significantly enhanced fine-grained video understanding, primarily driven by multimodal large language models (MLLMs). With superior multimodal comprehension and reasoning abilities, VTG approaches based on MLLMs (VTG-MLLMs) are gradually surpassing traditional fine-tuned methods. They not only achieve competitive performance but also excel in generalization across zero-shot, multi-task, and multi-domain settings. Despite extensive surveys on general video-language understanding, comprehensive reviews specifically addressing VTG-MLLMs remain scarce. To fill this gap, this survey systematically examines current research on VTG-MLLMs through a three-dimensional taxonomy: 1) the functional roles of MLLMs, highlighting their architectural significance; 2) training paradigms, analyzing strategies for temporal reasoning and task adaptation; and 3) video feature processing techniques, which determine spatiotemporal representation effectiveness. We further discuss benchmark datasets, evaluation protocols, and summarize empirical findings. Finally, we identify existing limitations and propose promising research directions. For additional resources and details, readers are encouraged to visit our repository at https://github.com/ki-lw/Awesome-MLLMs-for-Video-Temporal-Grounding.
Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 145 of its 180 references that are in the map. First posted on arXiv 7 Aug 2025.
Builds on
145 of 180 references.
Closest ancestors · 23
Same direction, newest first
- 2025MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding27 May
- 2025Slow-Fast Architecture for Video Multi-Modal Large Language Models2 Apr
- 2025SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capability18 Mar
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding17 Mar
- 2025AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding16 Mar
- 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding11 Mar
- 2025LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding14 Jan
- 2025Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models14 Jan
- 2024CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding16 Dec
- 2024VideoLights: Feature Refinement and Cross-Task Alignment Transformer for Joint Video Highlight Detection and Moment Retrieval2 Dec
- 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability27 Nov
- 2024ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videos22 Nov
11 more
- 2024LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrieval21 Nov
- 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning25 Oct
- 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models9 Aug
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning25 Apr
- 2024LITA: Language Instructed Temporal-Localization Assistant27 Mar
- 2024Efficient Temporal Extrapolation of Multimodal Large Language Models with Temporal Grounding Bridge25 Feb
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun
- 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning27 Feb
Other references · 122
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Visual Instruction Tuningcited by 7,869
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
38 more
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
- 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
- 2023VideoChat: Chat-Centric Video Understandingcited by 857
- 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
- 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
- 2017Dense-Captioning Events in Videoscited by 793
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
- 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
- 2023Baichuan 2: Open Large-scale Language Modelscited by 670
- 2017Localizing Moments in Video with Natural Languagecited by 663
- 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
- 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 619
- 2017TALL: Temporal Activity Localization via Language Querycited by 584
- 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
- 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
- 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 537
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
- 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
- 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
- 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 388
- 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
- 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
- 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
- 2023VTimeLLM: Empower LLM to Grasp Video Momentscited by 255
50 most cited of 122 All 122 (JSON)
Built on by
Citing papers per year: 2025: 7; 2026: 13.
20 papers.
Sort:
By area:
- 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 28
- 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 24
- 2025Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoningcited by 14
- 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
- 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioningcited by 8
- 2025ChronusOmni: Improving Time Awareness of Omni Large Language Modelscited by 6
- 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMscited by 4
- 2025FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensingcited by 3
- 2025When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactionscited by 3
- 2026Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Groundingcited by 3
- 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigmscited by 3
- 2026LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streamscited by 2
8 more
- 2026Pixelis: Reasoning in Pixels, from Seeing to Actingcited by 1
- 2026The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentationcited by 1
- 2026MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cuescited by 1
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
- 2026Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillationcited by 1
- 2026Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompt: The 1st Winner for 5th PVUW MOSE Challengecited by 0
- 2026Static and Dynamic Graph Alignment Network for Temporal Video Groundingcited by 0
- 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
- 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs9 Sep 2026
- 2026Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation11 Aug 2026
- 2026LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams16 Jun 2026
- 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning8 Jun 2026
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding5 Jun 2026
- 2026MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues21 May 2026
- 2026Static and Dynamic Graph Alignment Network for Temporal Video Grounding1 May 2026
- 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms10 Apr 2026
- 2026The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation1 Apr 2026
- 2026Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompt: The 1st Winner for 5th PVUW MOSE Challenge1 Apr 2026
- 2026Pixelis: Reasoning in Pixels, from Seeing to Acting26 Mar 2026
- 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs19 Mar 2026
8 more
- 2026Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding5 Mar 2026
- 2025Active Perception Agent for Omnimodal Audio-Video Understanding29 Dec 2025
- 2025Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning17 Dec 2025
- 2025ChronusOmni: Improving Time Awareness of Omni Large Language Models10 Dec 2025
- 2025When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions20 Oct 2025
- 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs1 Oct 2025
- 2025FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensing28 Sep 2025
- 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoning22 Sep 2025
- 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioningcited by 8
- 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMscited by 4
- 2025FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensingcited by 3
- 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigmscited by 3
- 2026MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cuescited by 1
- 2026CapRL++: Unified Reinforcement Learning with Verifiable Rewards for Dense Image and Video Captioning8 Jun 2026
- 2026MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues21 May 2026
- 2026How Should Video LLMs Output Time? An Analysis of Efficient Temporal Grounding Paradigms10 Apr 2026
- 2025Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs1 Oct 2025
- 2025FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensing28 Sep 2025
- 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 24
- 2025ChronusOmni: Improving Time Awareness of Omni Large Language Modelscited by 6
- 2025When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactionscited by 3
- 2026Pixelis: Reasoning in Pixels, from Seeing to Actingcited by 1
- 2026The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentationcited by 1
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understandingcited by 1
- 2026Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompt: The 1st Winner for 5th PVUW MOSE Challengecited by 0
- 2026Static and Dynamic Graph Alignment Network for Temporal Video Groundingcited by 0
- 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
- 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs9 Sep 2026
- 2026Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding5 Jun 2026
- 2026Static and Dynamic Graph Alignment Network for Temporal Video Grounding1 May 2026
- 2026The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation1 Apr 2026
- 2026Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompt: The 1st Winner for 5th PVUW MOSE Challenge1 Apr 2026
- 2026Pixelis: Reasoning in Pixels, from Seeing to Acting26 Mar 2026
- 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs19 Mar 2026
- 2025ChronusOmni: Improving Time Awareness of Omni Large Language Models10 Dec 2025
- 2025When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions20 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs
- 2026GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Grounding
- 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding
- 2025Unleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Grounding
- 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMs
- 2026OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding
- 2026UniversalVTG: A Universal and Lightweight Foundation Model for Video Temporal Grounding
- 2025ExpVG: Investigating the Design Space of Visual Grounding in Multimodal Large Language Model