# Sentence Specified Dynamic Video Thumbnail Generation

arXiv: https://arxiv.org/abs/1908.04052 · HTML: https://mlsanity.com/paper/1908.04052/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2019.

## Built on by (most cited first, 8 of 8)

- [UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2203.12745/) (2022)
- [QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries](https://mlsanity.com/paper/2107.09609/) (2021)
- [TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2401.02309/) (2024)
- [Let Me Finish My Sentence: Video Temporal Grounding with Holistic Text Understanding](https://mlsanity.com/paper/2410.13598/) (2024)
- [Video Summarization Overview](https://mlsanity.com/paper/2210.11707/) (2022)
- [Exploring Domain Incremental Video Highlights Detection with the LiveFood Benchmark](https://mlsanity.com/paper/2209.05166/) (2022)
- [LTC-GIF: Attracting More Clicks on Feature-length Sports Videos](https://mlsanity.com/paper/2201.09077/) (2022)
- [Character-focused Video Thumbnail Retrieval](https://mlsanity.com/paper/2204.06563/) (2022)

## Abstract (arXiv)

With the tremendous growth of videos over the Internet, video thumbnails, providing video content previews, are becoming increasingly crucial to influencing users' online searching experiences. Conventional video thumbnails are generated once purely based on the visual characteristics of videos, and then displayed as requested. Hence, such video thumbnails, without considering the users' searching intentions, cannot provide a meaningful snapshot of the video contents that users concern. In this paper, we define a distinctively new task, namely sentence specified dynamic video thumbnail generation, where the generated thumbnails not only provide a concise preview of the original video contents but also dynamically relate to the users' searching intentions with semantic correspondences to the users' query sentences. To tackle such a challenging task, we propose a novel graph convolved video thumbnail pointer (GTP). Specifically, GTP leverages a sentence specified video graph convolutional network to model both the sentence-video semantic interaction and the internal video relationships incorporated with the sentence information, based on which a temporal conditioned pointer network is then introduced to sequentially generate the sentence specified video thumbnails. Moreover, we annotate a new dataset based on ActivityNet Captions for the proposed new task, which consists of 10,000+ video-sentence pairs with each accompanied by an annotated sentence specified video thumbnail. We demonstrate that our proposed GTP outperforms several baseline methods on the created dataset, and thus believe that our initial results along with the release of the new dataset will inspire further research on sentence specified dynamic video thumbnail generation. Dataset and code are available at https://github.com/yytzsy/GTP.