# Weakly-Supervised Video Moment Retrieval via Semantic Completion Network

arXiv: https://arxiv.org/abs/1911.08199 · HTML: https://mlsanity.com/paper/1911.08199/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2019.

## Built on by (most cited first, 30 of 69)

- [Can I Trust Your Answer? Visually Grounded Video Question Answering](https://mlsanity.com/paper/2309.01327/) (2023)
- [TubeDETR: Spatio-Temporal Video Grounding with Transformers](https://mlsanity.com/paper/2203.16434/) (2022)
- [Interventional Video Grounding with Dual Contrastive Learning](https://mlsanity.com/paper/2106.11013/) (2021)
- [HawkEye: Training Video-Text LLMs for Grounding Text in Videos](https://mlsanity.com/paper/2403.10228/) (2024)
- [Video Corpus Moment Retrieval with Contrastive Learning](https://mlsanity.com/paper/2105.06247/) (2021)
- [Natural Language Video Localization: A Revisit in Span-based Question Answering Framework](https://mlsanity.com/paper/2102.13558/) (2021)
- [Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval](https://mlsanity.com/paper/2209.11572/) (2022)
- [You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos](https://mlsanity.com/paper/2303.07863/) (2023)
- [Cross-Sentence Temporal and Semantic Relations in Video Activity Localisation](https://mlsanity.com/paper/2107.11443/) (2021)
- [Hierarchical Local-Global Transformer for Temporal Sentence Grounding](https://mlsanity.com/paper/2208.14882/) (2022)
- [Weakly Supervised Temporal Adjacent Network for Language Grounding](https://mlsanity.com/paper/2106.16136/) (2021)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [Zero-shot Natural Language Video Localization](https://mlsanity.com/paper/2110.00428/) (2021)
- [Unsupervised Temporal Video Grounding with Deep Semantic Clustering](https://mlsanity.com/paper/2201.05307/) (2022)
- [A Survey on Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2109.08039/) (2021)
- [Hypotheses Tree Building for One-Shot Temporal Sentence Localization](https://mlsanity.com/paper/2301.01871/) (2023)
- [Parallel Attention Network with Sequence Matching for Video Grounding](https://mlsanity.com/paper/2105.08481/) (2021)
- [Language-free Training for Zero-shot Video Grounding](https://mlsanity.com/paper/2210.12977/) (2022)
- [End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding](https://mlsanity.com/paper/2203.08013/) (2022)
- [Video Moment Retrieval from Text Queries via Single Frame Annotation](https://mlsanity.com/paper/2204.09409/) (2022)
- [Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding](https://mlsanity.com/paper/2301.09071/) (2023)
- [A Survey on Video Moment Localization](https://mlsanity.com/paper/2306.07515/) (2023)
- [Progressive Localization Networks for Language-based Moment Localization](https://mlsanity.com/paper/2102.01282/) (2021)
- [Constraint and Union for Partially-Supervised Temporal Sentence Grounding](https://mlsanity.com/paper/2302.09850/) (2023)
- [Grounding-Prompter: Prompting LLM with Multimodal Information for Temporal Sentence Grounding in Long Videos](https://mlsanity.com/paper/2312.17117/) (2023)
- [SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/2310.05241/) (2023)
- [Towards Debiasing Temporal Sentence Grounding in Video](https://mlsanity.com/paper/2111.04321/) (2021)
- [Weakly-Supervised Temporal Article Grounding](https://mlsanity.com/paper/2210.12444/) (2022)
- [STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding](https://mlsanity.com/paper/2502.20678/) (2025)
- [Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos](https://mlsanity.com/paper/2208.01954/) (2022)

## Abstract (arXiv)

Video moment retrieval is to search the moment that is most relevant to the given natural language query. Existing methods are mostly trained in a fully-supervised setting, which requires the full annotations of temporal boundary for each query. However, manually labeling the annotations is actually time-consuming and expensive. In this paper, we propose a novel weakly-supervised moment retrieval framework requiring only coarse video-level annotations for training. Specifically, we devise a proposal generation module that aggregates the context information to generate and score all candidate proposals in one single pass. We then devise an algorithm that considers both exploitation and exploration to select top-K proposals. Next, we build a semantic completion module to measure the semantic similarity between the selected proposals and query, compute reward and provide feedbacks to the proposal generation module for scoring refinement. Experiments on the ActivityCaptions and Charades-STA demonstrate the effectiveness of our proposed method.