# Exploring Global Diversity and Local Context for Video Summarization

arXiv: https://arxiv.org/abs/2201.11345 · HTML: https://mlsanity.com/paper/2201.11345/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 27 Jan 2022.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Sequence to Sequence -- Video to Text](https://mlsanity.com/paper/1505.00487/) (2015)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward](https://mlsanity.com/paper/1801.00054/) (2017)
- [Video Summarization Using Fully Convolutional Sequence Networks](https://mlsanity.com/paper/1805.10538/) (2018)
- [Summarizing Videos with Attention](https://mlsanity.com/paper/1812.01969/) (2018)
- [Video Summarization with Attention-Based Encoder-Decoder Networks](https://mlsanity.com/paper/1708.09545/) (2017)
- [Rethinking the Evaluation of Video Summaries](https://mlsanity.com/paper/1903.11328/) (2019)
- [Less is More: Learning Highlight Detection from Video Duration](https://mlsanity.com/paper/1903.00859/) (2019)
- [MINI-Net: Multiple Instance Ranking Network for Video Highlight Detection](https://mlsanity.com/paper/2007.09833/) (2020)
- [Video2GIF: Automatic Generation of Animated GIFs from Video](https://mlsanity.com/paper/1605.04850/) (2016)
- [Discriminative Feature Learning for Unsupervised Video Summarization](https://mlsanity.com/paper/1811.09791/) (2018)
- [Video Summarization by Learning from Unpaired Data](https://mlsanity.com/paper/1805.12174/) (2018)
- [SumGraph: Video Summarization via Recursive Graph Modeling](https://mlsanity.com/paper/2007.08809/) (2020)
- [Exploring global diverse attention via pairwise temporal relation for video summarization](https://mlsanity.com/paper/2009.10942/) (2020)
- [Cycle-SUM: Cycle-consistent Adversarial LSTM Networks for Unsupervised Video Summarization](https://mlsanity.com/paper/1904.08265/) (2019)
- [Hierarchical Multimodal Transformer to Summarize Videos](https://mlsanity.com/paper/2109.10559/) (2021)

## Built on by (most cited first, 1 of 1)

- [SELF-VS: Self-supervised Encoding Learning For Video Summarization](https://mlsanity.com/paper/2303.15993/) (2023)

## Abstract (arXiv)

Video summarization aims to automatically generate a diverse and concise summary which is useful in large-scale video processing. Most of the methods tend to adopt self-attention mechanism across video frames, which fails to model the diversity of video frames. To alleviate this problem, we revisit the pairwise similarity measurement in self-attention mechanism and find that the existing inner-product affinity leads to discriminative features rather than diversified features. In light of this phenomenon, we propose global diverse attention which uses the squared Euclidean distance instead to compute the affinities. Moreover, we model the local contextual information by novel local contextual attention to remove the redundancy in the video. By combining these two attention mechanisms, a video SUMmarization model with Diversified Contextual Attention scheme is developed, namely SUM-DCA. Extensive experiments are conducted on benchmark data sets to verify the effectiveness and the superiority of SUM-DCA in terms of F-score and rank-based evaluation without any bells and whistles.