mlsanityData to 7 Oct 2026

Temporal Sentence Grounding in Videos: A Survey and Future Directions

arXiv 2201.08071 ↗First posted 20 Jan 2022cs.CV cs.AI cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

Temporal sentence grounding in videos (TSGV), aka natural language video localization (NLVL) or video moment retrieval (VMR), aims to retrieve a temporal moment that semantically corresponds to a language query from an untrimmed video. Connecting computer vision and natural language, TSGV has drawn significant attention from researchers in both communities. This survey attempts to provide a summary of fundamental concepts in TSGV and current research status, as well as future research directions. As the background, we present a common structure of functional components in TSGV, in a tutorial style: from feature extraction from raw video and language query, to answer prediction of the target moment. Then we review the techniques for multimodal understanding and interaction, which is the key focus of TSGV for effective alignment between the two modalities. We construct a taxonomy of TSGV techniques and elaborate the methods in different categories with their strengths and weaknesses. Lastly, we discuss issues with the current TSGV research and share our insights about promising research directions.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 154 of its 313 references that are in the map. First posted on arXiv 20 Jan 2022.

Builds on

154 of 313 references.

References · 154

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  9. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  10. 2020End-to-End Object Detection with Transformerscited by 7,384
  11. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  12. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
38 more
  1. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  2. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  3. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  4. 2017Dense-Captioning Events in Videoscited by 789
  5. 2017Localizing Moments in Video with Natural Languagecited by 660
  6. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 596
  7. 2017TALL: Temporal Activity Localization via Language Querycited by 581
  8. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  9. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  10. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  11. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  12. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  13. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  14. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  15. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  16. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  17. 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 327
  18. 2015Skip-Thought Vectorscited by 321
  19. 2018Deep Audio-Visual Speech Recognitioncited by 318
  20. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  21. 2014Caffe: Convolutional Architecture for Fast Feature Embeddingcited by 287
  22. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 283
  23. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  24. 2021TransVG: End-to-End Visual Grounding with Transformerscited by 269
  25. 2020ActBERT: Learning Global-Local Video-Text Representationscited by 267
  26. 2017Temporal Action Detection with Structured Segment Networkscited by 265
  27. 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 248
  28. 2016SoundNet: Learning Sound Representations from Unlabeled Videocited by 244
  29. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 234
  30. 2020TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrievalcited by 215
  31. 2020Local-Global Video-Text Interactions for Temporal Groundingcited by 188
  32. 2020Dense Regression Network for Video Groundingcited by 176
  33. 2020Support-set bottlenecks for video-text representation learningcited by 170
  34. 2021BEVT: BERT Pretraining of Video Transformerscited by 169
  35. 2017R-C3D: Region Convolutional 3D Network for Temporal Activity Detectioncited by 161
  36. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  37. 2019Simple BERT Models for Relation Extraction and Semantic Role Labelingcited by 154
  38. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 153

50 most cited of 154 All 154 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 12; 2025: 8; 2026: 10.

37 papers.

Sort:

By area:

  1. 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 138
  2. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 80
  3. 2024Toward Visual Grounding: A Surveycited by 52
  4. 2022Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Videocited by 43
  5. 2023Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challengescited by 42
  6. 2023MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformercited by 40
  7. 2026Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillationcited by 32
  8. 2024Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detectioncited by 25
  9. 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 24
  10. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 19
  11. 2024Weakly Supervised Gaussian Contrastive Grounding with Large Multimodal Models for Video Question Answeringcited by 14
  12. 2025Cross-modal Causal Relation Alignment for Video Question Groundingcited by 12
25 more
  1. 2024Beyond Uncertainty: Evidential Deep Learning for Robust Video Temporal Groundingcited by 10
  2. 2024GPTSee: Enhancing Moment Retrieval and Highlight Detection via Description-Based Similarity Featurescited by 6
  3. 2023Cross-modal Contrastive Learning with Asymmetric Co-attention Network for Video Moment Retrievalcited by 4
  4. 2024Temporally Grounding Instructional Diagrams in Unconstrained Videoscited by 4
  5. 2023Efficient Temporal Sentence Grounding in Videos with Multi-Teacher Knowledge Distillationcited by 3
  6. 2023Mug-STAN: Adapting Image-Language Pretrained Models for General Video Understandingcited by 3
  7. 2024UAL-Bench: The First Comprehensive Unusual Activity Localization Benchmarkcited by 3
  8. 2025The Devil is in the Spurious Correlations: Boosting Moment Retrieval with Dynamic Learningcited by 3
  9. 2023DiffusionVMR: Diffusion Model for Joint Video Moment Retrieval and Highlight Detectioncited by 2
  10. 2025Learning to Refuse: Refusal-Aware Reinforcement Fine-Tuning for Hard-Irrelevant Queries in Video Temporal Groundingcited by 2
  11. 2024Temporal Contrastive Learning for Video Temporal Reasoning in Large Vision-Language Modelscited by 1
  12. 2025Invert4TVG: A Temporal Video Grounding Framework with Inversion Tasks Preserving Action Understanding Abilitycited by 1
  13. 2026Retrievit: In-context Retrieval Capabilities of Transformers, State Space Models, and Hybrid Architecturescited by 1
  14. 2026T2SGrid: Temporal-to-Spatial Gridification for Video Temporal Groundingcited by 1
  15. 2024Video sentence grounding with temporally global textual knowledgecited by 0
  16. 2024A Simple and Effective Temporal Grounding Pipeline for Basketball Broadcast Footagecited by 0
  17. 2025Collaborative Temporal Consistency Learning for Point-supervised Natural Language Video Localizationcited by 0
  18. 2025ViewBridge: Curriculum Knowledge Distillation for Activity View-Invariance Under Extreme Viewpoint Changescited by 0
  19. 2026GenSpan: Generation-Calibrated Motion Span Priors for Multi-Verb Video Corpus Moment Retrievalcited by 0
  20. 2026Static and Dynamic Graph Alignment Network for Temporal Video Groundingcited by 0
  21. 2026Multi-proposal Collaboration and Multi-task Training for Weakly-supervised Video Moment Retrievalcited by 0
  22. 2026Mining Multi-Modality Spatio-Temporal Cues for Video Important Person Identificationcited by 0
  23. 2026GIRL-DETR: Gradient-Isolated Reinforcement Learning for Video Moment Retrievalcited by 0
  24. 2026Conformal Coverage Guarantees for Any Video Temporal Groundercited by 0
  25. 2026Multi-Agent Self-Improving Reinforcement Learning for Video Reasoningcited by 0