# Vid-Morp: Video Moment Retrieval Pretraining from Unlabeled Videos in the Wild

arXiv: https://arxiv.org/abs/2412.00811 · HTML: https://mlsanity.com/paper/2412.00811/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 58 references that are in the map.
- First posted on arXiv 1 Dec 2024.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [UniVTG: Towards Unified Video-Language Temporal Grounding](https://mlsanity.com/paper/2307.16715/) (2023)
- [Local-Global Video-Text Interactions for Temporal Grounding](https://mlsanity.com/paper/2004.07514/) (2020)
- [Dense Regression Network for Video Grounding](https://mlsanity.com/paper/2004.03545/) (2020)
- [To Find Where You Talk: Temporal Sentence Localization in Video with Attention Based Location Regression](https://mlsanity.com/paper/1804.07014/) (2018)
- [MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment](https://mlsanity.com/paper/1812.00087/) (2018)
- [Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Prediction](https://mlsanity.com/paper/1909.05010/) (2019)
- [Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videos](https://mlsanity.com/paper/1906.02497/) (2019)
- [QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries](https://mlsanity.com/paper/2107.09609/) (2021)
- [UnLoc: A Unified Framework for Video Localization Tasks](https://mlsanity.com/paper/2308.11062/) (2023)
- [Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning](https://mlsanity.com/paper/2203.13049/) (2022)
- [Weakly Supervised Dense Event Captioning in Videos](https://mlsanity.com/paper/1812.03849/) (2018)
- [Tree-Structured Policy based Progressive Reinforcement Learning for Temporally Language Grounding in Video](https://mlsanity.com/paper/2001.06680/) (2020)
- [A Closer Look at Temporal Sentence Grounding in Videos: Dataset and Metric](https://mlsanity.com/paper/2101.09028/) (2021)
- [Zero-shot Natural Language Video Localization](https://mlsanity.com/paper/2110.00428/) (2021)
- [Exploiting Temporal Relationships in Video Moment Localization with Natural Language](https://mlsanity.com/paper/1908.03846/) (2019)
- [BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videos](https://mlsanity.com/paper/2312.00083/) (2023)
- [VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs](https://mlsanity.com/paper/2409.20365/) (2024)
- [Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval](https://mlsanity.com/paper/2312.12155/) (2023)
- [Language-free Training for Zero-shot Video Grounding](https://mlsanity.com/paper/2210.12977/) (2022)
- [Zero-Shot Video Moment Retrieval from Frozen Vision-Language Models](https://mlsanity.com/paper/2309.00661/) (2023)
- [VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPT](https://mlsanity.com/paper/2403.02076/) (2024)

## Built on by (most cited first, 5 of 5)

- [A Survey on Video Temporal Grounding with Multimodal Large Language Model](https://mlsanity.com/paper/2508.10922/) (2025)
- [A Lightweight Moment Retrieval System with Global Re-Ranking and Robust Adaptive Bidirectional Temporal Search](https://mlsanity.com/paper/2504.09298/) (2025)
- [Towards One-to-Many Temporal Grounding](https://mlsanity.com/paper/2606.06294/) (2026)
- [Watch, Remember, Reason: Human-View Video Understanding with MLLMs](https://mlsanity.com/paper/2606.07433/) (2026)
- [Beyond Caption-Based Queries for Video Moment Retrieval](https://mlsanity.com/paper/2603.02363/) (2026)

## Abstract (arXiv)

Given a natural language query, video moment retrieval aims to localize the described temporal moment in an untrimmed video. A major challenge of this task is its heavy dependence on labor-intensive annotations for training. Unlike existing works that directly train models on manually curated data, we propose a novel paradigm to reduce annotation costs: pretraining the model on unlabeled, real-world videos. To support this, we introduce Video Moment Retrieval Pretraining (Vid-Morp), a large-scale dataset collected with minimal human intervention, consisting of over 50K videos captured in the wild and 200K pseudo annotations. Direct pretraining on these imperfect pseudo annotations, however, presents significant challenges, including mismatched sentence-video pairs and imprecise temporal boundaries. To address these issues, we propose the ReCorrect algorithm, which comprises two main phases: semantics-guided refinement and memory-consensus correction. The semantics-guided refinement enhances the pseudo labels by leveraging semantic similarity with video frames to clean out unpaired data and make initial adjustments to temporal boundaries. In the following memory-consensus correction phase, a memory bank tracks the model predictions, progressively correcting the temporal boundaries based on consensus within the memory. Comprehensive experiments demonstrate ReCorrect's strong generalization abilities across multiple downstream settings. Zero-shot ReCorrect achieves over 75% and 80% of the best fully-supervised performance on two benchmarks, while unsupervised ReCorrect reaches about 85% on both. The code, dataset, and pretrained models are available at https://github.com/baopj/Vid-Morp.