# Transforming Multi-Concept Attention into Video Summarization

arXiv: https://arxiv.org/abs/2006.01410 · HTML: https://mlsanity.com/paper/2006.01410/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2020.

## Built on by (most cited first, 9 of 9)

- [V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning](https://mlsanity.com/paper/2404.12353/) (2024)
- [Supervised Video Summarization via Multiple Feature Sets with Parallel Attention](https://mlsanity.com/paper/2104.11530/) (2021)
- [Hierarchical Multimodal Transformer to Summarize Videos](https://mlsanity.com/paper/2109.10559/) (2021)
- [Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward](https://mlsanity.com/paper/2209.12164/) (2022)
- [Video Summarization Overview](https://mlsanity.com/paper/2210.11707/) (2022)
- [Contrastive Losses Are Natural Criteria for Unsupervised Video Summarization](https://mlsanity.com/paper/2211.10056/) (2022)
- [MHSCNet: A Multimodal Hierarchical Shot-aware Convolutional Network for Video Summarization](https://mlsanity.com/paper/2204.08352/) (2022)
- [FastPerson: Enhancing Video Learning through Effective Video Summarization that Preserves Linguistic and Visual Contexts](https://mlsanity.com/paper/2403.17727/) (2024)
- [SELF-VS: Self-supervised Encoding Learning For Video Summarization](https://mlsanity.com/paper/2303.15993/) (2023)

## Abstract (arXiv)

Video summarization is among challenging tasks in computer vision, which aims at identifying highlight frames or shots over a lengthy video input. In this paper, we propose an novel attention-based framework for video summarization with complex video data. Unlike previous works which only apply attention mechanism on the correspondence between frames, our multi-concept video self-attention (MC-VSA) model is presented to identify informative regions across temporal and concept video features, which jointly exploit context diversity over time and space for summarization purposes. Together with consistency between video and summary enforced in our framework, our model can be applied to both labeled and unlabeled data, making our method preferable to real-world applications. Extensive and complete experiments on two benchmarks demonstrate the effectiveness of our model both quantitatively and qualitatively, and confirms its superiority over the stateof-the-arts.