mlsanityData to 7 Oct 2026

Zero-shot Natural Language Video Localization

arXiv 2110.00428 ↗First posted 29 Aug 2021cs.CL cs.AI cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

Understanding videos to localize moments with natural language often requires large expensive annotated video regions paired with language queries. To eliminate the annotation costs, we make a first attempt to train a natural language video localization model in zero-shot manner. Inspired by unsupervised image captioning setup, we merely require random text corpora, unlabeled video collections, and an off-the-shelf object detector to train a model. With the unpaired data, we propose to generate pseudo-supervision of candidate temporal regions and corresponding query sentences, and develop a simple NLVL model to train with the pseudo-supervision. Our empirical validations show that the proposed pseudo-supervised method outperforms several baseline approaches and a number of methods using stronger supervision on Charades-STA and ActivityNet-Captions.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 29 Aug 2021.

Builds on

40 of 59 references.

References · 40

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  3. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  4. 2017Non-local Neural Networkscited by 2,290
  5. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  6. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  7. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  8. 2017Dense-Captioning Events in Videoscited by 789
  9. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  10. 2017Localizing Moments in Video with Natural Languagecited by 660
  11. 2017Towards Automatic Learning of Procedures from Web Instructional Videoscited by 616
  12. 2017TALL: Temporal Activity Localization via Language Querycited by 581
28 more
  1. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  2. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  3. 2020ActBERT: Learning Global-Local Video-Text Representationscited by 267
  4. 2018End-to-End Dense Video Captioning with Masked Transformercited by 214
  5. 2020Local-Global Video-Text Interactions for Temporal Groundingcited by 188
  6. 2020Dense Regression Network for Video Groundingcited by 176
  7. 2017Unsupervised Neural Machine Translationcited by 148
  8. 2018MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustmentcited by 145
  9. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  10. 2018Neural Baby Talkcited by 107
  11. 2019Weakly Supervised Video Moment Retrieval From Text Queriescited by 85
  12. 2019Proposal-free Temporal Moment Localization of a Natural-Language Query in Video using Guided Attentioncited by 81
  13. 2018Unsupervised Image Captioningcited by 79
  14. 2019Weakly-Supervised Video Moment Retrieval via Semantic Completion Networkcited by 69
  15. 2020Counting Out Time: Class Agnostic Video Repetition Counting in the Wildcited by 67
  16. 2018Weakly Supervised Dense Event Captioning in Videoscited by 58
  17. 2019Streamlined Dense Video Captioningcited by 57
  18. 2019Towards Unsupervised Image Captioning with Shared Multimodal Embeddingscited by 54
  19. 2019WSLLN: Weakly Supervised Natural Language Localization Networkscited by 36
  20. 2015Transductive Zero-Shot Action Recognition by Word-Vector Embeddingcited by 31
  21. 2020Look Closer to Ground Better: Weakly-Supervised Temporal Grounding of Sentence in Videocited by 29
  22. 2015Objects2action: Classifying and localizing actions without any video examplecited by 26
  23. 2015Temporal Localization of Fine-Grained Actions in Videos by Domain Transfer from Web Imagescited by 19
  24. 2018Decoupled Novel Object Captionercited by 15
  25. 2017Attributes2Classname: A discriminative model for attribute-based unsupervised zero-shot learningcited by 12
  26. 2017Learning to Learn from Noisy Web Videoscited by 5
  27. 2019Semantic Fisher Scores for Task Transfer: Using Objects to Classify Scenescited by 3
  28. 2020DORi: Discovering Object Relationship for Moment Localization of a Natural-Language Query in Videocited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 8; 2024: 9; 2025: 3; 2026: 8.

34 papers.

Sort:

By area:

  1. 2023Knowing Where to Focus: Event-aware Transformer for Video Groundingcited by 79
  2. 2024ChatVTG: Video Temporal Grounding via Chat with Video Dialogue Large Language Modelscited by 54
  3. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  4. 2022Pseudo-Q: Generating Pseudo Language Queries for Visual Groundingcited by 33
  5. 2024Training-free Video Temporal Grounding using Large-scale Pre-trained Modelscited by 28
  6. 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 24
  7. 2022Language-free Training for Zero-shot Video Groundingcited by 22
  8. 2023Zero-Shot Video Moment Retrieval from Frozen Vision-Language Modelscited by 22
  9. 2021UBoCo : Unsupervised Boundary Contrastive Learning for Generic Event Boundary Detectioncited by 20
  10. 2024VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPTcited by 19
  11. 2023A Survey on Video Moment Localizationcited by 18
  12. 2023Dense Video Captioning: A Survey of Techniques, Datasets and Evaluation Protocolscited by 17
22 more
  1. 2024Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentationcited by 13
  2. 2025Enrich and Detect: Video Temporal Grounding with Multimodal LLMscited by 12
  3. 2025TAG: A Simple Yet Effective Temporal-Aware Approach for Zero-Shot Video Temporal Groundingcited by 7
  4. 2022Modal-specific Pseudo Query Generation for Video Corpus Moment Retrievalcited by 6
  5. 2024Localizing Events in Videos with Multimodal Queriescited by 6
  6. 2024Vid-Morp: Video Moment Retrieval Pretraining from Unlabeled Videos in the Wildcited by 5
  7. 2026GranAlign: Granularity-Aware Alignment Framework for Zero-Shot Video Moment Retrievalcited by 5
  8. 2023Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Momentcited by 4
  9. 2023Commonsense for Zero-Shot Natural Language Video Localizationcited by 4
  10. 2023Faster Video Moment Retrieval with Point-Level Supervisioncited by 3
  11. 2024Generative Video Diffusion for Unseen Novel Semantic Video Moment Retrievalcited by 2
  12. 2025ResidualViT for Efficient Temporally Dense Video Encodingcited by 2
  13. 2022Dyadic Movement Synchrony Estimation Under Privacy-preserving Conditionscited by 1
  14. 2023Language-Conditioned Change-point Detection to Identify Sub-Tasks in Robotics Domainscited by 1
  15. 2026DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Groundingcited by 1
  16. 2026Music-to-Dance Generation via Atomic Movementscited by 1
  17. 2024AutoTVG: A New Vision-language Pre-training Paradigm for Temporal Video Groundingcited by 0
  18. 2026All Eyes on the Workflow: Automated and Efficient Event Discovery from Video Streamscited by 0
  19. 2026Multi-proposal Collaboration and Multi-task Training for Weakly-supervised Video Moment Retrievalcited by 0
  20. 2026DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Modelscited by 0
  21. 2026Mitigating Modality and Language-Style Gaps for Zero-Shot Video Moment Retrievalcited by 0
  22. 2026DSE-VTG: Dual-Side Enhancement for Training-Free Video Temporal Groundingcited by 0