# Faster Video Moment Retrieval with Point-Level Supervision

arXiv: https://arxiv.org/abs/2305.14017 · HTML: https://mlsanity.com/paper/2305.14017/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Dense-Captioning Events in Videos](https://mlsanity.com/paper/1705.00754/) (2017)
- [Localizing Moments in Video with Natural Language](https://mlsanity.com/paper/1708.01641/) (2017)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/1912.03590/) (2019)
- [Local-Global Video-Text Interactions for Temporal Grounding](https://mlsanity.com/paper/2004.07514/) (2020)
- [Weakly Supervised Video Moment Retrieval From Text Queries](https://mlsanity.com/paper/1904.03282/) (2019)
- [Weakly-Supervised Video Moment Retrieval via Semantic Completion Network](https://mlsanity.com/paper/1911.08199/) (2019)
- [SF-Net: Single-Frame Supervision for Temporal Action Localization](https://mlsanity.com/paper/2003.06845/) (2020)
- [Where are the Blobs: Counting by Localization with Point Supervision](https://mlsanity.com/paper/1807.09856/) (2018)
- [Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2108.05029/) (2021)
- [A Self-Training Approach for Point-Supervised Object Detection and Counting in Crowds](https://mlsanity.com/paper/2007.12831/) (2020)
- [Zero-shot Natural Language Video Localization](https://mlsanity.com/paper/2110.00428/) (2021)
- [Action Recognition from Single Timestamp Supervision in Untrimmed Videos](https://mlsanity.com/paper/1904.04689/) (2019)
- [Unsupervised Temporal Video Grounding with Deep Semantic Clustering](https://mlsanity.com/paper/2201.05307/) (2022)
- [Support-Set Based Cross-Supervision for Video Grounding](https://mlsanity.com/paper/2108.10576/) (2021)
- [Video Moment Retrieval from Text Queries via Single Frame Annotation](https://mlsanity.com/paper/2204.09409/) (2022)
- [Spot On: Action Localization from Pointly-Supervised Proposals](https://mlsanity.com/paper/1604.07602/) (2016)
- [Self-supervised Learning for Semi-supervised Temporal Language Grounding](https://mlsanity.com/paper/2109.11475/) (2021)

## Built on by (most cited first, 3 of 3)

- [Reversed in Time: A Novel Temporal-Emphasized Benchmark for Cross-Modal Video-Text Retrieval](https://mlsanity.com/paper/2412.19178/) (2024)
- [Show and Guide: Instructional-Plan Grounded Vision and Language Model](https://mlsanity.com/paper/2409.19074/) (2024)
- [Collaborative Temporal Consistency Learning for Point-supervised Natural Language Video Localization](https://mlsanity.com/paper/2503.17651/) (2025)

## Abstract (arXiv)

Video Moment Retrieval (VMR) aims at retrieving the most relevant events from an untrimmed video with natural language queries. Existing VMR methods suffer from two defects: (1) massive expensive temporal annotations are required to obtain satisfying performance; (2) complicated cross-modal interaction modules are deployed, which lead to high computational cost and low efficiency for the retrieval process. To address these issues, we propose a novel method termed Cheaper and Faster Moment Retrieval (CFMR), which well balances the retrieval accuracy, efficiency, and annotation cost for VMR. Specifically, our proposed CFMR method learns from point-level supervision where each annotation is a single frame randomly located within the target moment. It is 6 times cheaper than the conventional annotations of event boundaries. Furthermore, we also design a concept-based multimodal alignment mechanism to bypass the usage of cross-modal interaction modules during the inference process, remarkably improving retrieval efficiency. The experimental results on three widely used VMR benchmarks demonstrate the proposed CFMR method establishes new state-of-the-art with point-level supervision. Moreover, it significantly accelerates the retrieval speed with more than 100 times FLOPs compared to existing approaches with point-level supervision.