# Sentence Guided Temporal Modulation for Dynamic Video Thumbnail Generation

arXiv: https://arxiv.org/abs/2008.13362 · HTML: https://mlsanity.com/paper/2008.13362/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2020.

## Built on by (most cited first, 3 of 3)

- [UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2203.12745/) (2022)
- [Video Summarization Overview](https://mlsanity.com/paper/2210.11707/) (2022)
- [Joint Moment Retrieval and Highlight Detection Via Natural Language Queries](https://mlsanity.com/paper/2305.04961/) (2023)

## Abstract (arXiv)

We consider the problem of sentence specified dynamic video thumbnail generation. Given an input video and a user query sentence, the goal is to generate a video thumbnail that not only provides the preview of the video content, but also semantically corresponds to the sentence. In this paper, we propose a sentence guided temporal modulation (SGTM) mechanism that utilizes the sentence embedding to modulate the normalized temporal activations of the video thumbnail generation network. Unlike the existing state-of-the-art method that uses recurrent architectures, we propose a non-recurrent framework that is simple and allows much more parallelization. Extensive experiments and analysis on a large-scale dataset demonstrate the effectiveness of our framework.