# TRIM: A Self-Supervised Video Summarization Framework Maximizing Temporal Relative Information and Representativeness

arXiv: https://arxiv.org/abs/2506.20588 · HTML: https://mlsanity.com/paper/2506.20588/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward](https://mlsanity.com/paper/1801.00054/) (2017)
- [CLIP-It! Language-Guided Video Summarization](https://mlsanity.com/paper/2107.00650/) (2021)
- [Video Summarization Using Fully Convolutional Sequence Networks](https://mlsanity.com/paper/1805.10538/) (2018)
- [Summarizing Videos with Attention](https://mlsanity.com/paper/1812.01969/) (2018)
- [Align and Attend: Multimodal Summarization with Dual Contrastive Losses](https://mlsanity.com/paper/2303.07284/) (2023)
- [Rethinking the Evaluation of Video Summaries](https://mlsanity.com/paper/1903.11328/) (2019)
- [Hierarchical Recurrent Neural Network for Video Summarization](https://mlsanity.com/paper/1904.12251/) (2019)
- [Discriminative Feature Learning for Unsupervised Video Summarization](https://mlsanity.com/paper/1811.09791/) (2018)
- [Exploring global diverse attention via pairwise temporal relation for video summarization](https://mlsanity.com/paper/2009.10942/) (2020)
- [Supervised Video Summarization via Multiple Feature Sets with Parallel Attention](https://mlsanity.com/paper/2104.11530/) (2021)
- [CSTA: CNN-based Spatiotemporal Attention for Video Summarization](https://mlsanity.com/paper/2405.11905/) (2024)
- [Progressive Video Summarization via Multimodal Self-supervised Learning](https://mlsanity.com/paper/2201.02494/) (2022)
- [Hierarchical Multimodal Transformer to Summarize Videos](https://mlsanity.com/paper/2109.10559/) (2021)
- [Weakly Supervised Video Summarization by Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2001.05864/) (2020)
- [Query Twice: Dual Mixture Attention Meta Learning for Video Summarization](https://mlsanity.com/paper/2008.08360/) (2020)
- [Video Joint Modelling Based on Hierarchical Transformer for Co-summarization](https://mlsanity.com/paper/2112.13478/) (2021)
- [Masked Autoencoder for Unsupervised Video Summarization](https://mlsanity.com/paper/2306.01395/) (2023)

## Built on by (most cited first, 2 of 2)

- [TRIMMER: A New Paradigm for Video Summarization through Self-Supervised Reinforcement Learning](https://mlsanity.com/paper/2605.01659/) (2026)
- [SMI: Efficient Self-Supervised Learning via Mutual-Information-Inspired Dependency Optimization](https://mlsanity.com/paper/2606.08332/) (2026)

## Abstract (arXiv)

The increasing ubiquity of video content and the corresponding demand for efficient access to meaningful information have elevated video summarization and video highlights as a vital research area. However, many state-of-the-art methods depend heavily either on supervised annotations or on attention-based models, which are computationally expensive and brittle in the face of distribution shifts that hinder cross-domain applicability across datasets. We introduce a pioneering self-supervised video summarization model that captures both spatial and temporal dependencies without the overhead of attention, RNNs, or transformers. Our framework integrates a novel set of Markov process-driven loss metrics and a two-stage self supervised learning paradigm that ensures both performance and efficiency. Our approach achieves state-of-the-art performance on the SUMME and TVSUM datasets, outperforming all existing unsupervised methods. It also rivals the best supervised models, demonstrating the potential for efficient, annotation-free architectures. This paves the way for more generalizable video summarization techniques and challenges the prevailing reliance on complex architectures.