# Unsupervised Video Summarization via Iterative Training and Simplified GAN

arXiv: https://arxiv.org/abs/2311.03745 · HTML: https://mlsanity.com/paper/2311.03745/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 7 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [UniVTG: Towards Unified Video-Language Temporal Grounding](https://mlsanity.com/paper/2307.16715/) (2023)
- [Query-Dependent Video Representation for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2303.13874/) (2023)
- [Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN](https://mlsanity.com/paper/1803.04831/) (2018)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward](https://mlsanity.com/paper/1801.00054/) (2017)
- [Video Summarization Using Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.06072/) (2021)
- [Hierarchical Video-Moment Retrieval and Step-Captioning](https://mlsanity.com/paper/2303.16406/) (2023)
- [Video2GIF: Automatic Generation of Animated GIFs from Video](https://mlsanity.com/paper/1605.04850/) (2016)
- [Cross-category Video Highlight Detection via Set-based Learning](https://mlsanity.com/paper/2108.11770/) (2021)
- [Discriminative Feature Learning for Unsupervised Video Summarization](https://mlsanity.com/paper/1811.09791/) (2018)
- [A Coding Framework and Benchmark towards Low-Bitrate Video Understanding](https://mlsanity.com/paper/2202.02813/) (2022)
- [Unsupervised Video Summarization via Multi-source Features](https://mlsanity.com/paper/2105.12532/) (2021)

## Built on by (most cited first, 3 of 3)

- [AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interaction](https://mlsanity.com/paper/2506.18472/) (2025)
- [Reinforcement Learning for Unsupervised Video Summarization with Reward Generator Training](https://mlsanity.com/paper/2407.04258/) (2024)
- [Identity-Centric Video Summarization via Hierarchical Fusion of Biometric, Appearance, and 3D Body Features](https://mlsanity.com/paper/2609.25837/) (2026)

## Abstract (arXiv)

This paper introduces a new, unsupervised method for automatic video summarization using ideas from generative adversarial networks but eliminating the discriminator, having a simple loss function, and separating training of different parts of the model. An iterative training strategy is also applied by alternately training the reconstructor and the frame selector for multiple iterations. Furthermore, a trainable mask vector is added to the model in summary generation during training and evaluation. The method also includes an unsupervised model selection algorithm. Results from experiments on two public datasets (SumMe and TVSum) and four datasets we created (Soccer, LoL, MLB, and ShortMLB) demonstrate the effectiveness of each component on the model performance, particularly the iterative training strategy. Evaluations and comparisons with the state-of-the-art methods highlight the advantages of the proposed method in performance, stability, and training efficiency.