# Multi-proposal Collaboration and Multi-task Training for Weakly-supervised Video Moment Retrieval

arXiv: https://arxiv.org/abs/2605.14838 · HTML: https://mlsanity.com/paper/2605.14838/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 14 May 2026.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [Weakly Supervised Video Moment Retrieval From Text Queries](https://mlsanity.com/paper/1904.03282/) (2019)
- [Weakly-Supervised Video Moment Retrieval via Semantic Completion Network](https://mlsanity.com/paper/1911.08199/) (2019)
- [Weakly Supervised Dense Event Captioning in Videos](https://mlsanity.com/paper/1812.03849/) (2018)
- [VLANet: Video-Language Alignment Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/2008.10238/) (2020)
- [LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/1909.13784/) (2019)
- [Cross-Sentence Temporal and Semantic Relations in Video Activity Localisation](https://mlsanity.com/paper/2107.11443/) (2021)
- [Weakly Supervised Temporal Adjacent Network for Language Grounding](https://mlsanity.com/paper/2106.16136/) (2021)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [WSLLN: Weakly Supervised Natural Language Localization Networks](https://mlsanity.com/paper/1909.00239/) (2019)
- [Zero-shot Natural Language Video Localization](https://mlsanity.com/paper/2110.00428/) (2021)
- [Weakly-Supervised Multi-Level Attentional Reconstruction Network for Grounding Textual Queries in Videos](https://mlsanity.com/paper/2003.07048/) (2020)
- [Look Closer to Ground Better: Weakly-Supervised Temporal Grounding of Sentence in Video](https://mlsanity.com/paper/2001.09308/) (2020)
- [Regularized Two-Branch Proposal Networks for Weakly-Supervised Moment Retrieval in Videos](https://mlsanity.com/paper/2008.08257/) (2020)
- [A Survey on Video Moment Localization](https://mlsanity.com/paper/2306.07515/) (2023)
- [SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/2310.05241/) (2023)
- [Counterfactual Cross-modality Reasoning for Weakly Supervised Video Moment Localization](https://mlsanity.com/paper/2308.05648/) (2023)

## Abstract (arXiv)

This study focuses on weakly-supervised Video Moment Retrieval (VMR), aiming to identify a moment semantically similar to the given query within an untrimmed video using only video-level correspondences, without relying on temporal annotations during training. Previous methods either aggregate predictions for all instances in the video, or indirectly address the task by proposing reconstructions for the query. However, these methods often produce low-quality temporal proposals, struggle with distinguishing misaligned moments in the same video, or lack stability due to a reliance on a single auxiliary task. To address these limitations, we present a novel weakly-supervised method called Multi-proposal Collaboration and Multi-task Training (MCMT). Initially, we generate multiple proposals and derive corresponding learnable Gaussian masks from them. These masks are then combined to create a high-quality positive sample mask, highlighting video clips most relevant to the query. Concurrently, we classify other clips in the same video as the easy negative sample and the entire video as the hard negative sample. During training, we introduce forward and inverse masked query reconstruction tasks to impose more substantial constraints on the network, promoting more robust and stable retrieval performance. Extensive experiments on two standard benchmarks affirm the effectiveness of the proposed method in VMR.