# PR-Net: Preference Reasoning for Personalized Video Highlight Detection

arXiv: https://arxiv.org/abs/2109.01799 · HTML: https://mlsanity.com/paper/2109.01799/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 4 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward](https://mlsanity.com/paper/1801.00054/) (2017)
- [Reinforcement Knowledge Graph Reasoning for Explainable Recommendation](https://mlsanity.com/paper/1906.05237/) (2019)
- [Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences](https://mlsanity.com/paper/1902.06236/) (2019)
- [Video Summarization Using Fully Convolutional Sequence Networks](https://mlsanity.com/paper/1805.10538/) (2018)
- [Improving Question Answering by Commonsense-Based Pre-Training](https://mlsanity.com/paper/1809.03568/) (2018)
- [Less is More: Learning Highlight Detection from Video Duration](https://mlsanity.com/paper/1903.00859/) (2019)
- [Hierarchical Recurrent Neural Network for Video Summarization](https://mlsanity.com/paper/1904.12251/) (2019)
- [MINI-Net: Multiple Instance Ranking Network for Video Highlight Detection](https://mlsanity.com/paper/2007.09833/) (2020)
- [Video2GIF: Automatic Generation of Animated GIFs from Video](https://mlsanity.com/paper/1605.04850/) (2016)
- [Query-Focused Extractive Video Summarization](https://mlsanity.com/paper/1607.05177/) (2016)
- [CAFE: Coarse-to-Fine Neural Symbolic Reasoning for Explainable Recommendation](https://mlsanity.com/paper/2010.15620/) (2020)
- [Summary Transfer: Exemplar-based Subset Selection for Video Summarization](https://mlsanity.com/paper/1603.03369/) (2016)
- [Adaptive Video Highlight Detection by Learning from User History](https://mlsanity.com/paper/2007.09598/) (2020)
- [PHD-GIFs: Personalized Highlight Detection for Automatic GIF Creation](https://mlsanity.com/paper/1804.06604/) (2018)
- [Collaborative Summarization of Topic-Related Videos](https://mlsanity.com/paper/1706.03114/) (2017)
- [A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360 Video](https://mlsanity.com/paper/1801.10312/) (2018)
- [Ridiculously Fast Shot Boundary Detection with Fully Convolutional Neural Networks](https://mlsanity.com/paper/1705.08214/) (2017)

## Built on by (most cited first, 6 of 6)

- [Towards Label-free Scene Understanding by Vision Foundation Models](https://mlsanity.com/paper/2306.03899/) (2023)
- [Finding the Right Moment: Human-Assisted Trailer Creation via Task Composition](https://mlsanity.com/paper/2111.08774/) (2021)
- [Probing Visual-Audio Representation for Video Highlight Detection via Hard-Pairs Guided Contrastive Learning](https://mlsanity.com/paper/2206.10157/) (2022)
- [Show Me What I Like: Detecting User-Specific Video Highlights Using Content-Based Multi-Head Attention](https://mlsanity.com/paper/2207.08352/) (2022)
- [ContentCTR: Frame-level Live Streaming Click-Through Rate Prediction with Multimodal Transformer](https://mlsanity.com/paper/2306.14392/) (2023)
- [A Multimodal Transformer for Live Streaming Highlight Prediction](https://mlsanity.com/paper/2407.12002/) (2024)

## Abstract (arXiv)

Personalized video highlight detection aims to shorten a long video to interesting moments according to a user's preference, which has recently raised the community's attention. Current methods regard the user's history as holistic information to predict the user's preference but negating the inherent diversity of the user's interests, resulting in vague preference representation. In this paper, we propose a simple yet efficient preference reasoning framework (PR-Net) to explicitly take the diverse interests into account for frame-level highlight prediction. Specifically, distinct user-specific preferences for each input query frame are produced, presented as the similarity weighted sum of history highlights to the corresponding query frame. Next, distinct comprehensive preferences are formed by the user-specific preferences and a learnable generic preference for more overall highlight measurement. Lastly, the degree of highlight and non-highlight for each query frame is calculated as semantic similarity to its comprehensive and non-highlight preferences, respectively. Besides, to alleviate the ambiguity due to the incomplete annotation, a new bi-directional contrastive loss is proposed to ensure a compact and differentiable metric space. In this way, our method significantly outperforms state-of-the-art methods with a relative improvement of 12% in mean accuracy precision.