# Towards Micro-video Thumbnail Selection via a Multi-label Visual-semantic Embedding Model

arXiv: https://arxiv.org/abs/2202.02930 · HTML: https://mlsanity.com/paper/2202.02930/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 7 Feb 2022.

## Builds on (most cited first, 10 of 10)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [GRCN: Graph-Refined Convolutional Network for Multimedia Recommendation with Implicit Feedback](https://mlsanity.com/paper/2111.02036/) (2021)
- [To Click or Not To Click: Automatic Selection of Beautiful Thumbnails from Videos](https://mlsanity.com/paper/1609.01388/) (2016)
- [Describing Multimedia Content using Attention-based Encoder--Decoder Networks](https://mlsanity.com/paper/1507.01053/) (2015)
- [Diversified Visual Attention Networks for Fine-Grained Object Classification](https://mlsanity.com/paper/1606.08572/) (2016)
- [Learning Deep Latent Spaces for Multi-Label Classification](https://mlsanity.com/paper/1707.00418/) (2017)
- [Building Emotional Machines: Recognizing Image Emotions through Deep Neural Networks](https://mlsanity.com/paper/1705.07543/) (2017)
- [Hierarchical User Intent Graph Network forMultimedia Recommendation](https://mlsanity.com/paper/2110.14925/) (2021)
- [A Benchmark Dataset for Micro-video Thumbnail Selection](https://mlsanity.com/paper/2112.14958/) (2021)

## Abstract (arXiv)

The thumbnail, as the first sight of a micro-video, plays a pivotal role in attracting users to click and watch. While in the real scenario, the more the thumbnails satisfy the users, the more likely the micro-videos will be clicked. In this paper, we aim to select the thumbnail of a given micro-video that meets most users` interests. Towards this end, we present a multi-label visual-semantic embedding model to estimate the similarity between the pair of each frame and the popular topics that users are interested in. In this model, the visual and textual information is embedded into a shared semantic space, whereby the similarity can be measured directly, even the unseen words. Moreover, to compare the frame to all words from the popular topics, we devise an attention embedding space associated with the semantic-attention projection. With the help of these two embedding spaces, the popularity score of a frame, which is defined by the sum of similarity scores over the corresponding visual information and popular topic pairs, is achieved. Ultimately, we fuse the visual representation score and the popularity score of each frame to select the attractive thumbnail for the given micro-video. Extensive experiments conducted on a real-world dataset have well-verified that our model significantly outperforms several state-of-the-art baselines.