# Discriminative Feature Learning for Unsupervised Video Summarization

arXiv: https://arxiv.org/abs/1811.09791 · HTML: https://mlsanity.com/paper/1811.09791/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2018.

## Built on by (most cited first, 30 of 37)

- [Video Summarization Using Deep Neural Networks: A Survey](https://mlsanity.com/paper/2101.06072/) (2021)
- [Align and Attend: Multimodal Summarization with Dual Contrastive Losses](https://mlsanity.com/paper/2303.07284/) (2023)
- [GPT2MVS: Generative Pre-trained Transformer-2 for Multi-modal Video Summarization](https://mlsanity.com/paper/2104.12465/) (2021)
- [Progressive Video Summarization via Multimodal Self-supervised Learning](https://mlsanity.com/paper/2201.02494/) (2022)
- [IntentVizor: Towards Generic Query Guided Interactive Video Summarization](https://mlsanity.com/paper/2109.14834/) (2021)
- [Scaling Up Video Summarization Pretraining with Large Language Models](https://mlsanity.com/paper/2404.03398/) (2024)
- [UnweaveNet: Unweaving Activity Stories](https://mlsanity.com/paper/2112.10194/) (2021)
- [Video Joint Modelling Based on Hierarchical Transformer for Co-summarization](https://mlsanity.com/paper/2112.13478/) (2021)
- [Unsupervised Domain Adaptation for Video Semantic Segmentation](https://mlsanity.com/paper/2107.11052/) (2021)
- [Unsupervised Video Summarization via Multi-source Features](https://mlsanity.com/paper/2105.12532/) (2021)
- [Conditional Modeling Based Automatic Video Summarization](https://mlsanity.com/paper/2311.12159/) (2023)
- [Cluster-based Video Summarization with Temporal Context Awareness](https://mlsanity.com/paper/2404.04511/) (2024)
- [Contrastive Losses Are Natural Criteria for Unsupervised Video Summarization](https://mlsanity.com/paper/2211.10056/) (2022)
- [VVS: Video-to-Video Retrieval with Irrelevant Frame Suppression](https://mlsanity.com/paper/2303.08906/) (2023)
- [Spiking Variational Graph Representation Inference for Video Summarization](https://mlsanity.com/paper/2508.15389/) (2025)
- [Multimodal Frame-Scoring Transformer for Video Summarization](https://mlsanity.com/paper/2207.01814/) (2022)
- [VideoSAGE: Video Summarization with Graph Representation Learning](https://mlsanity.com/paper/2404.10539/) (2024)
- [Use of Affective Visual Information for Summarization of Human-Centric Videos](https://mlsanity.com/paper/2107.03783/) (2021)
- [ERA: Entity Relationship Aware Video Summarization with Wasserstein GAN](https://mlsanity.com/paper/2109.02625/) (2021)
- [Unsupervised Video Summarization via Iterative Training and Simplified GAN](https://mlsanity.com/paper/2311.03745/) (2023)
- [Language-Guided Self-Supervised Video Summarization Using Text Semantic Matching Considering the Diversity of the Video](https://mlsanity.com/paper/2405.08890/) (2024)
- [HierSum: A Global and Local Attention Mechanism for Video Summarization](https://mlsanity.com/paper/2504.18689/) (2025)
- [MHSCNet: A Multimodal Hierarchical Shot-aware Convolutional Network for Video Summarization](https://mlsanity.com/paper/2204.08352/) (2022)
- [Masked Autoencoder for Unsupervised Video Summarization](https://mlsanity.com/paper/2306.01395/) (2023)
- [TRIM: A Self-Supervised Video Summarization Framework Maximizing Temporal Relative Information and Representativeness](https://mlsanity.com/paper/2506.20588/) (2025)
- [TripleSumm: Adaptive Triple-Modality Fusion for Video Summarization](https://mlsanity.com/paper/2603.01169/) (2026)
- [Exploring Global Diversity and Local Context for Video Summarization](https://mlsanity.com/paper/2201.11345/) (2022)
- [Scene Summarization: Clustering Scene Videos into Spatially Diverse Frames](https://mlsanity.com/paper/2311.17940/) (2023)
- [Early Exit and Multi Stage Knowledge Distillation in VLMs for Video Summarization](https://mlsanity.com/paper/2504.21831/) (2025)
- [TRIMMER: A New Paradigm for Video Summarization through Self-Supervised Reinforcement Learning](https://mlsanity.com/paper/2605.01659/) (2026)

## Abstract (arXiv)

In this paper, we address the problem of unsupervised video summarization that automatically extracts key-shots from an input video. Specifically, we tackle two critical issues based on our empirical observations: (i) Ineffective feature learning due to flat distributions of output importance scores for each frame, and (ii) training difficulty when dealing with long-length video inputs. To alleviate the first problem, we propose a simple yet effective regularization loss term called variance loss. The proposed variance loss allows a network to predict output scores for each frame with high discrepancy which enables effective feature learning and significantly improves model performance. For the second problem, we design a novel two-stream network named Chunk and Stride Network (CSNet) that utilizes local (chunk) and global (stride) temporal view on the video features. Our CSNet gives better summarization results for long-length videos compared to the existing methods. In addition, we introduce an attention mechanism to handle the dynamic information in videos. We demonstrate the effectiveness of the proposed methods by conducting extensive ablation studies and show that our final model achieves new state-of-the-art results on two benchmark datasets.