Masked Autoencoder for Unsupervised Video Summarization
Not on the map. Why?
Abstract (arXiv)
Summarizing a video requires a diverse understanding of the video, ranging from recognizing scenes to evaluating how much each frame is essential enough to be selected as a summary. Self-supervised learning (SSL) is acknowledged for its robustness and flexibility to multiple downstream tasks, but the video SSL has not shown its value for dense understanding tasks like video summarization. We claim an unsupervised autoencoder with sufficient self-supervised learning does not need any extra downstream architecture design or fine-tuning weights to be utilized as a video summarization model. The proposed method to evaluate the importance score of each frame takes advantage of the reconstruction score of the autoencoder's decoder. We evaluate the method in major unsupervised video summarization benchmarks to show its effectiveness under various experimental settings.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 51 of its 71 references that are in the map. First posted on arXiv 2 Jun 2023.
Builds on
51 of 71 references.
References · 51
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
38 more
- 2014Going Deeper with Convolutionscited by 4,241
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
- 2018SlowFast Networks for Video Recognitioncited by 1,867
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
- 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
- 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
- 2016Deep Networks with Stochastic Depthcited by 979
- 2021Multiscale Vision Transformerscited by 897
- 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
- 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
- 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
- 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
- 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
- 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
- 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 234
- 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
- 2021PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformerscited by 204
- 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
- 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
- 2016Video Summarization with Long Short-term Memorycited by 142
- 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
- 2017Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Rewardcited by 105
- 2021Video Summarization Using Deep Neural Networks: A Surveycited by 85
- 2020RSPNet: Relative Speed Perception for Unsupervised Video Representation Learningcited by 83
- 2018Video Summarization Using Fully Convolutional Sequence Networkscited by 71
- 2018Summarizing Videos with Attentioncited by 69
- 2021Spatially Consistent Representation Learningcited by 64
- 2019Rethinking the Evaluation of Video Summariescited by 62
- 2019Hierarchical Recurrent Neural Network for Video Summarizationcited by 44
- 2021Long-Short Temporal Contrastive Learning of Video Transformerscited by 42
- 2018Discriminative Feature Learning for Unsupervised Video Summarizationcited by 37
- 2018Video Summarization by Learning from Unpaired Datacited by 36
- 2020SumGraph: Video Summarization via Recursive Graph Modelingcited by 32
- 2015Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoderscited by 30
- 2020Exploring global diverse attention via pairwise temporal relation for video summarizationcited by 29
- 2019Cycle-SUM: Cycle-consistent Adversarial LSTM Networks for Unsupervised Video Summarizationcited by 24
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.