# DORi: Discovering Object Relationship for Moment Localization of a Natural-Language Query in Video

arXiv: https://arxiv.org/abs/2010.06260 · HTML: https://mlsanity.com/paper/2010.06260/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 3 of 3)

- [Zero-shot Natural Language Video Localization](https://mlsanity.com/paper/2110.00428/) (2021)
- [Learning Sample Importance for Cross-Scenario Video Temporal Grounding](https://mlsanity.com/paper/2201.02848/) (2022)
- [Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding](https://mlsanity.com/paper/2209.13359/) (2022)

## Abstract (arXiv)

This paper studies the task of temporal moment localization in a long untrimmed video using natural language query. Given a query sentence, the goal is to determine the start and end of the relevant segment within the video. Our key innovation is to learn a video feature embedding through a language-conditioned message-passing algorithm suitable for temporal moment localization which captures the relationships between humans, objects and activities in the video. These relationships are obtained by a spatial sub-graph that contextualizes the scene representation using detected objects and human features conditioned in the language query. Moreover, a temporal sub-graph captures the activities within the video through time. Our method is evaluated on three standard benchmark datasets, and we also introduce YouCookII as a new benchmark for this task. Experiments show our method outperforms state-of-the-art methods on these datasets, confirming the effectiveness of our approach.