# Cycle-SUM: Cycle-consistent Adversarial LSTM Networks for Unsupervised Video Summarization

arXiv: https://arxiv.org/abs/1904.08265 · HTML: https://mlsanity.com/paper/1904.08265/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2019.

## Built on by (most cited first, 24 of 24)

- [CLIP-It! Language-Guided Video Summarization](https://mlsanity.com/paper/2107.00650/) (2021)
- [Video Summarization Using Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.06072/) (2021)
- [Align and Attend: Multimodal Summarization with Dual Contrastive Losses](https://mlsanity.com/paper/2303.07284/) (2023)
- [TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency](https://mlsanity.com/paper/2208.06773/) (2022)
- [GPT2MVS: Generative Pre-trained Transformer-2 for Multi-modal Video Summarization](https://mlsanity.com/paper/2104.12465/) (2021)
- [Video Summarization through Reinforcement Learning with a 3D Spatio-Temporal U-Net](https://mlsanity.com/paper/2106.10528/) (2021)
- [Video Summarization with Large Language Models](https://mlsanity.com/paper/2504.11199/) (2025)
- [UnweaveNet: Unweaving Activity Stories](https://mlsanity.com/paper/2112.10194/) (2021)
- [Unsupervised Video Summarization via Multi-source Features](https://mlsanity.com/paper/2105.12532/) (2021)
- [Unsupervised Modality-Transferable Video Highlight Detection with Representation Activation Sequence Learning](https://mlsanity.com/paper/2403.09401/) (2024)
- [How Good is a Video Summary? A New Benchmarking Dataset and Evaluation Framework Towards Realistic Video Summarization](https://mlsanity.com/paper/2101.10514/) (2021)
- [Conditional Modeling Based Automatic Video Summarization](https://mlsanity.com/paper/2311.12159/) (2023)
- [Spiking Variational Graph Representation Inference for Video Summarization](https://mlsanity.com/paper/2508.15389/) (2025)
- [Multimodal Frame-Scoring Transformer for Video Summarization](https://mlsanity.com/paper/2207.01814/) (2022)
- [ERA: Entity Relationship Aware Video Summarization with Wasserstein GAN](https://mlsanity.com/paper/2109.02625/) (2021)
- [Prompts to Summaries: Zero-Shot Language-Guided Video Summarization with Large Language and Video Models](https://mlsanity.com/paper/2506.10807/) (2025)
- [MHSCNet: A Multimodal Hierarchical Shot-aware Convolutional Network for Video Summarization](https://mlsanity.com/paper/2204.08352/) (2022)
- [Masked Autoencoder for Unsupervised Video Summarization](https://mlsanity.com/paper/2306.01395/) (2023)
- [TripleSumm: Adaptive Triple-Modality Fusion for Video Summarization](https://mlsanity.com/paper/2603.01169/) (2026)
- [Exploring Global Diversity and Local Context for Video Summarization](https://mlsanity.com/paper/2201.11345/) (2022)
- [Scene Summarization: Clustering Scene Videos into Spatially Diverse Frames](https://mlsanity.com/paper/2311.17940/) (2023)
- [Self-Attention Based Generative Adversarial Networks For Unsupervised Video Summarization](https://mlsanity.com/paper/2307.08145/) (2023)
- [Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization](https://mlsanity.com/paper/2509.05604/) (2025)
- [Context-Aware Pseudo-Label Scoring for Zero-Shot Video Summarization](https://mlsanity.com/paper/2510.17501/) (2025)

## Abstract (arXiv)

In this paper, we present a novel unsupervised video summarization model that requires no manual annotation. The proposed model termed Cycle-SUM adopts a new cycle-consistent adversarial LSTM architecture that can effectively maximize the information preserving and compactness of the summary video. It consists of a frame selector and a cycle-consistent learning based evaluator. The selector is a bi-direction LSTM network that learns video representations that embed the long-range relationships among video frames. The evaluator defines a learnable information preserving metric between original video and summary video and "supervises" the selector to identify the most informative frames to form the summary video. In particular, the evaluator is composed of two generative adversarial networks (GANs), in which the forward GAN is learned to reconstruct original video from summary video while the backward GAN learns to invert the processing. The consistency between the output of such cycle learning is adopted as the information preserving metric for video summarization. We demonstrate the close relation between mutual information maximization and such cycle learning procedure. Experiments on two video summarization benchmark datasets validate the state-of-the-art performance and superiority of the Cycle-SUM model over previous baselines.