mlsanityData to 7 Oct 2026

UniVTG: Towards Unified Video-Language Temporal Grounding

arXiv 2307.16715 ↗First posted 31 Jul 2023cs.CVcited by 189

Not on the map. Why?

Cited by 30 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Video Temporal Grounding (VTG), which aims to ground target clips from videos (such as consecutive intervals or disjoint shots) according to custom language queries (e.g., sentences or words), is key for video browsing on social media. Most methods in this direction develop taskspecific models that are trained with type-specific labels, such as moment retrieval (time interval) and highlight detection (worthiness curve), which limits their abilities to generalize to various VTG tasks and labels. In this paper, we propose to Unify the diverse VTG labels and tasks, dubbed UniVTG, along three directions: Firstly, we revisit a wide range of VTG labels and tasks and define a unified formulation. Based on this, we develop data annotation schemes to create scalable pseudo supervision. Secondly, we develop an effective and flexible grounding model capable of addressing each task and making full use of each label. Lastly, thanks to the unified framework, we are able to unlock temporal grounding pretraining from large-scale diverse labels and develop stronger grounding abilities e.g., zero-shot grounding. Extensive experiments on three tasks (moment retrieval, highlight detection and video summarization) across seven datasets (QVHighlights, Charades-STA, TACoS, Ego4D, YouTube Highlights, TVSum, and QFVS) demonstrate the effectiveness and flexibility of our proposed framework. The codes are available at https://github.com/showlab/UniVTG.

Key facts. Cited by 189 papers in the map (data through 7 Oct 2026). Builds on 45 of its 70 references that are in the map. First posted on arXiv 31 Jul 2023.

Builds on

45 of 70 references.

References · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  3. 2015Fast R-CNNcited by 2,930
  4. 2018SlowFast Networks for Video Recognitioncited by 1,862
  5. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  7. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  8. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  9. 2021Grounded Language-Image Pre-trainingcited by 1,147
  10. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  11. 2017Localizing Moments in Video with Natural Languagecited by 660
  12. 2017TALL: Temporal Activity Localization via Language Querycited by 581
33 more
  1. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  2. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 283
  3. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  4. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  5. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
  6. 2022Egocentric Video-Language Pretrainingcited by 231
  7. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  8. 2022Learning Video Representations from Large Language Modelscited by 204
  9. 2020Local-Global Video-Text Interactions for Temporal Groundingcited by 188
  10. 2020Dense Regression Network for Video Groundingcited by 176
  11. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
  12. 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
  13. 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
  14. 2016Video Summarization with Long Short-term Memorycited by 142
  15. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  16. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  17. 2022Learning Audio-Video Modalities from Image Captionscited by 80
  18. 2020TSP: Temporally-Sensitive Pretraining of Video Encoders for Localization Taskscited by 77
  19. 2019Less is More: Learning Highlight Detection from Video Durationcited by 60
  20. 2023NaQ: Leveraging Narrations as Queries to Supervise Episodic Memorycited by 46
  21. 2022LocVTP: Video-Text Pre-training for Temporal Localizationcited by 44
  22. 2017Query-Focused Video Summarization: Dataset, Evaluation, and A Memory Network Based Approachcited by 42
  23. 2020MINI-Net: Multiple Instance Ranking Network for Video Highlight Detectioncited by 42
  24. 2016Video2GIF: Automatic Generation of Animated GIFs from Videocited by 41
  25. 2020Boundary-sensitive Pre-training for Temporal Localization in Videoscited by 41
  26. 2021Cross-category Video Highlight Detection via Set-based Learningcited by 38
  27. 2020Learning Trailer Moments in Full-Length Moviescited by 37
  28. 2016To Click or Not To Click: Automatic Selection of Beautiful Thumbnails from Videoscited by 33
  29. 2022Unsupervised Pre-training for Temporal Action Localization Taskscited by 31
  30. 2015Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoderscited by 30
  31. 2022VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignmentcited by 20
  32. 2021IntentVizor: Towards Generic Query Guided Interactive Video Summarizationcited by 18
  33. 2020Convolutional Hierarchical Attention Network for Query-Focused Video Summarizationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 67; 2025: 68; 2026: 44.

189 papers.

Sort:

By area:

  1. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  2. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  3. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 214
  4. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 212
  5. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 203
  6. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  7. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoningcited by 122
  8. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  9. 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuningcited by 99
  10. 2024TRACE: Temporal Grounding Video LLM via Causal Event Modelingcited by 87
  11. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Abilitycited by 69
  12. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 68
38 more
  1. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  2. 2023Grounded Question-Answering in Long Egocentric Videoscited by 62
  3. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 61
  4. 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perceptioncited by 60
  5. 2024VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computationcited by 55
  6. 2024ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Modelscited by 54
  7. 2023Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Modelcited by 52
  8. 2024ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videoscited by 51
  9. 2024LongVALE: Vision-Audio-Language-Event Benchmark Towards Time-Aware Omni-Modal Perception of Long Videoscited by 49
  10. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 45
  11. 2024Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomalycited by 44
  12. 2024MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videoscited by 43
  13. 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videoscited by 43
  14. 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 41
  15. 2024SnAG: Scalable and Accurate Video Groundingcited by 36
  16. 2024Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillationcited by 36
  17. 2024R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Groundingcited by 34
  18. 2023Correlation-Guided Query-Dependency Calibration for Video Temporal Groundingcited by 31
  19. 2024Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrievalcited by 30
  20. 2023BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videoscited by 29
  21. 2024VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMscited by 29
  22. 2024Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignmentcited by 29
  23. 2025Universal Video Temporal Grounding with Generative Multi-modal Large Language Modelscited by 26
  24. 2024SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Groundingcited by 25
  25. 2024Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detectioncited by 25
  26. 2024FlashVTG: Feature Layering and Adaptive Score Handling Network for Video Temporal Groundingcited by 23
  27. 2024OmniVid: A Generative Framework for Universal Video Understandingcited by 22
  28. 2024Saliency-Guided DETR for Moment Retrieval and Highlight Detectioncited by 21
  29. 2024VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPTcited by 19
  30. 2024LinVT: Empower Your Image-level Large Language Model to Understand Videoscited by 19
  31. 2024Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Modelcited by 19
  32. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 19
  33. 2024ReVisionLLM: Recursive Vision-Language Model for Temporal Grounding in Hour-Long Videoscited by 18
  34. 2025Knowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Groundingcited by 17
  35. 2024Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motioncited by 16
  36. 2024Learning Video Context as Interleaved Multimodal Sequencescited by 16
  37. 2024EAGLE: Egocentric AGgregated Language-video Enginecited by 16
  38. 2025Impossible Videoscited by 16

50 most cited of 189 All 189 (JSON)