# VLANet: Video-Language Alignment Network for Weakly-Supervised Video Moment Retrieval

arXiv: https://arxiv.org/abs/2008.10238 · HTML: https://mlsanity.com/paper/2008.10238/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2020.

## Built on by (most cited first, 30 of 41)

- [Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval](https://mlsanity.com/paper/2209.11572/) (2022)
- [You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos](https://mlsanity.com/paper/2303.07863/) (2023)
- [Cross-Sentence Temporal and Semantic Relations in Video Activity Localisation](https://mlsanity.com/paper/2107.11443/) (2021)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [Unsupervised Temporal Video Grounding with Deep Semantic Clustering](https://mlsanity.com/paper/2201.05307/) (2022)
- [A Survey on Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2109.08039/) (2021)
- [Language-free Training for Zero-shot Video Grounding](https://mlsanity.com/paper/2210.12977/) (2022)
- [A Survey on Video Temporal Grounding with Multimodal Large Language Model](https://mlsanity.com/paper/2508.10922/) (2025)
- [Video Moment Retrieval from Text Queries via Single Frame Annotation](https://mlsanity.com/paper/2204.09409/) (2022)
- [Locate before Answering: Answer Guided Question Localization for Video Question Answering](https://mlsanity.com/paper/2210.02081/) (2022)
- [A Survey on Video Moment Localization](https://mlsanity.com/paper/2306.07515/) (2023)
- [Structured Co-reference Graph Attention for Video-grounded Dialogue](https://mlsanity.com/paper/2103.13361/) (2021)
- [FRAG: Frequency Adapting Group for Diffusion Video Editing](https://mlsanity.com/paper/2406.06044/) (2024)
- [SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/2310.05241/) (2023)
- [Weakly-Supervised Temporal Article Grounding](https://mlsanity.com/paper/2210.12444/) (2022)
- [Gaussian Mixture Proposals with Pull-Push Learning Scheme to Capture Diverse Events for Weakly Supervised Temporal Video Grounding](https://mlsanity.com/paper/2312.16388/) (2023)
- [Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding](https://mlsanity.com/paper/2403.11463/) (2024)
- [Rethinking Weakly-supervised Video Temporal Grounding From a Game Perspective](https://mlsanity.com/paper/2605.26441/) (2026)
- [Self-supervised Learning for Semi-supervised Temporal Language Grounding](https://mlsanity.com/paper/2109.11475/) (2021)
- [Counterfactual Cross-modality Reasoning for Weakly Supervised Video Moment Localization](https://mlsanity.com/paper/2308.05648/) (2023)
- [A Survey on Natural Language Video Localization](https://mlsanity.com/paper/2104.00234/) (2021)
- [Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval](https://mlsanity.com/paper/2210.12617/) (2022)
- [Localizing Events in Videos with Multimodal Queries](https://mlsanity.com/paper/2406.10079/) (2024)
- [Zero-Shot Dense Video Captioning by Jointly Optimizing Text and Moment](https://mlsanity.com/paper/2307.02682/) (2023)
- [EtC: Temporal Boundary Expand then Clarify for Weakly Supervised Video Grounding with Multimodal Large Language Model](https://mlsanity.com/paper/2312.02483/) (2023)
- [Commonsense for Zero-Shot Natural Language Video Localization](https://mlsanity.com/paper/2312.17429/) (2023)
- [Fine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding](https://mlsanity.com/paper/2210.11933/) (2022)
- [Occlusion-robust Stylization for Drawing-based 3D Animation](https://mlsanity.com/paper/2508.00398/) (2025)
- [Weakly Supervised Temporal Sentence Grounding via Positive Sample Mining](https://mlsanity.com/paper/2505.06557/) (2025)
- [Annotations Are Not All You Need: A Cross-modal Knowledge Transfer Network for Unsupervised Temporal Sentence Grounding](https://mlsanity.com/paper/2605.30742/) (2026)

## Abstract (arXiv)

Video Moment Retrieval (VMR) is a task to localize the temporal moment in untrimmed video specified by natural language query. For VMR, several methods that require full supervision for training have been proposed. Unfortunately, acquiring a large number of training videos with labeled temporal boundaries for each query is a labor-intensive process. This paper explores methods for performing VMR in a weakly-supervised manner (wVMR): training is performed without temporal moment labels but only with the text query that describes a segment of the video. Existing methods on wVMR generate multi-scale proposals and apply query-guided attention mechanisms to highlight the most relevant proposal. To leverage the weak supervision, contrastive learning is used which predicts higher scores for the correct video-query pairs than for the incorrect pairs. It has been observed that a large number of candidate proposals, coarse query representation, and one-way attention mechanism lead to blurry attention maps which limit the localization performance. To handle this issue, Video-Language Alignment Network (VLANet) is proposed that learns sharper attention by pruning out spurious candidate proposals and applying a multi-directional attention mechanism with fine-grained query representation. The Surrogate Proposal Selection module selects a proposal based on the proximity to the query in the joint embedding space, and thus substantially reduces candidate proposals which leads to lower computation load and sharper attention. Next, the Cascaded Cross-modal Attention module considers dense feature interactions and multi-directional attention flow to learn the multi-modal alignment. VLANet is trained end-to-end using contrastive loss which enforces semantically similar videos and queries to gather. The experiments show that the method achieves state-of-the-art performance on Charades-STA and DiDeMo datasets.