mlsanityData to 14 Oct 2026

Temporal Sentence Grounding in Streaming Videos

arXiv 2308.07102 ↗First posted 14 Aug 2023cs.CV cs.CL cs.MMcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper aims to tackle a novel task - Temporal Sentence Grounding in Streaming Videos (TSGSV). The goal of TSGSV is to evaluate the relevance between a video stream and a given sentence query. Unlike regular videos, streaming videos are acquired continuously from a particular source, and are always desired to be processed on-the-fly in many applications such as surveillance and live-stream analysis. Thus, TSGSV is challenging since it requires the model to infer without future frames and process long historical frames effectively, which is untouched in the early methods. To specifically address the above challenges, we propose two novel methods: (1) a TwinNet structure that enables the model to learn about upcoming events; and (2) a language-guided feature compressor that eliminates redundant visual frames and reinforces the frames that are relevant to the query. We conduct extensive experiments using ActivityNet Captions, TACoS, and MAD datasets. The results demonstrate the superiority of our proposed methods. A systematic ablation study also confirms their effectiveness.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 32 of its 68 references that are in the map. First posted on arXiv 14 Aug 2023.

Builds on

32 of 68 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2017Dense-Captioning Events in Videoscited by 793
  4. 2017Localizing Moments in Video with Natural Languagecited by 663
  5. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  6. 2019Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 285
  7. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  8. 2018To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regressioncited by 157
  9. 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
  10. 2021MAD: A Scalable Dataset for Language Grounding in Videos from Movie Audio Descriptionscited by 108
  11. 2021Long Short-Term Transformer for Online Action Detectioncited by 88
  12. 2021OadTR: Online Action Detection with Transformerscited by 83
20 more
  1. 2019ExCL: Extractive Clip Localization Using Natural Language Descriptionscited by 82
  2. 2018Temporal Recurrent Networks for Online Action Detectioncited by 80
  3. 2022Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generationcited by 67
  4. 2019Read, Watch, and Move: Reinforcement Learning for Temporally Grounding Natural Language Descriptions in Videoscited by 60
  5. 2021Natural Language Video Localization: A Revisit in Span-based Question Answering Frameworkcited by 55
  6. 2020Tree-Structured Policy based Progressive Reinforcement Learning for Temporally Language Grounding in Videocited by 47
  7. 2020Fine-grained Iterative Attention Network for TemporalLanguage Localization in Videoscited by 46
  8. 2021A Closer Look at Temporal Sentence Grounding in Videos: Dataset and Metriccited by 42
  9. 2019Learning to Discriminate Information for Online Action Detectioncited by 37
  10. 2023Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Trainingcited by 32
  11. 2020Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Languagecited by 29
  12. 2021A Survey on Temporal Sentence Grounding in Videoscited by 29
  13. 2022GateHUB: Gated History Unit with Background Suppression for Online Action Detectioncited by 27
  14. 2020Reinforcement Learning for Weakly Supervised Temporal Grounding of Natural Language in Untrimmed Videoscited by 23
  15. 2023Scanning Only Once: An End-to-end Framework for Fast Temporal Grounding in Long Videoscited by 23
  16. 2021Parallel Attention Network with Sequence Matching for Video Groundingcited by 22
  17. 2023Text-Visual Prompting for Efficient 2D Temporal Video Groundingcited by 18
  18. 2023HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendationcited by 10
  19. 2023Micro-video Tagging via Jointly Modeling Social Influence and Tag Relationcited by 3
  20. 2022AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanismcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 1.

8 papers.

  1. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 81
  2. 2025AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understandingcited by 48
  3. 2025Datasets and Recipes for Video Temporal Grounding via Reinforcement Learningcited by 19
  4. 2026Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanismcited by 9
  5. 2024M2-RAAP: A Multi-Modal Recipe for Advancing Adaptation-based Pre-training towards Effective and Efficient Zero-shot Video-text Retrievalcited by 5
  6. 2025Hierarchical Event Memory for Accurate and Low-latency Online Video Temporal Groundingcited by 5
  7. 2025Detector-Empowered Video Large Language Model for Efficient Spatio-Temporal Groundingcited by 1
  8. 2025OVG-HQ: Online Video Grounding with Hybrid-modal Queriescited by 0