# Text-based Localization of Moments in a Video Corpus

arXiv: https://arxiv.org/abs/2008.08716 · HTML: https://mlsanity.com/paper/2008.08716/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2020.

## Built on by (most cited first, 11 of 11)

- [Zero-Shot Temporal Action Detection via Vision-Language Prompting](https://mlsanity.com/paper/2207.08184/) (2022)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [PRVR: Partially Relevant Video Retrieval](https://mlsanity.com/paper/2208.12510/) (2022)
- [Multi-Modal Few-Shot Temporal Action Detection](https://mlsanity.com/paper/2211.14905/) (2022)
- [Video Moment Retrieval with Text Query Considering Many-to-Many Correspondence Using Potentially Relevant Pair](https://mlsanity.com/paper/2106.13566/) (2021)
- [Dual-task Mutual Reinforcing Embedded Joint Video Paragraph Retrieval and Grounding](https://mlsanity.com/paper/2411.17481/) (2024)
- [MamFusion: Multi-Mamba with Temporal Fusion for Partially Relevant Video Retrieval](https://mlsanity.com/paper/2506.03473/) (2025)
- [Who Can We Trust? Scope-Aware Video Moment Retrieval with Multi-Agent Conflict](https://mlsanity.com/paper/2511.00370/) (2025)
- [Topic-to-Timestamp Alignment by Constrained Evidence Selection](https://mlsanity.com/paper/2606.20890/) (2026)
- [DAEP: Difficulty-Aware Evidence Planning for Medical Video Corpus Temporal Answer Grounding](https://mlsanity.com/paper/2608.06869/) (2026)
- [Grounded Product Understanding in Livestream Videos](https://mlsanity.com/paper/2609.20508/) (2026)

## Abstract (arXiv)

Prior works on text-based video moment localization focus on temporally grounding the textual query in an untrimmed video. These works assume that the relevant video is already known and attempt to localize the moment on that relevant video only. Different from such works, we relax this assumption and address the task of localizing moments in a corpus of videos for a given sentence query. This task poses a unique challenge as the system is required to perform: (i) retrieval of the relevant video where only a segment of the video corresponds with the queried sentence, and (ii) temporal localization of moment in the relevant video based on sentence query. Towards overcoming this challenge, we propose Hierarchical Moment Alignment Network (HMAN) which learns an effective joint embedding space for moments and sentences. In addition to learning subtle differences between intra-video moments, HMAN focuses on distinguishing inter-video global semantic concepts based on sentence queries. Qualitative and quantitative results on three benchmark text-based video moment retrieval datasets - Charades-STA, DiDeMo, and ActivityNet Captions - demonstrate that our method achieves promising performance on the proposed task of temporal localization of moments in a corpus of videos.