mlsanityData to 7 Oct 2026

Video Summarization with Long Short-term Memory

arXiv 1605.08110 ↗First posted 26 May 2016cs.CV cs.LGcited by 142

Not on the map. Why?

Abstract (arXiv)

We propose a novel supervised learning technique for summarizing videos by automatically selecting keyframes or key subshots. Casting the problem as a structured prediction problem on sequential data, our main idea is to use Long Short-Term Memory (LSTM), a special type of recurrent neural networks to model the variable-range dependencies entailed in the task of video summarization. Our learning models attain the state-of-the-art results on two benchmark video datasets. Detailed analysis justifies the design of the models. In particular, we show that it is crucial to take into consideration the sequential structures in videos and model them. Besides advances in modeling techniques, we introduce techniques to address the need of a large number of annotated data for training complex learning models. There, our main idea is to exploit the existence of auxiliary annotated video datasets, albeit heterogeneous in visual styles and contents. Specifically, we show domain adaptation techniques can improve summarization by reducing the discrepancies in statistical properties across those datasets.

Key facts. Cited by 142 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 27; 2023: 30; 2024: 26; 2025: 23; 2026: 10.

142 papers.

Sort:

By area:

  1. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 399
  2. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 189
  3. 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
  4. 2022UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detectioncited by 148
  5. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  6. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 101
  7. 2021CLIP-It! Language-Guided Video Summarizationcited by 99
  8. 2021Video Summarization Using Deep Neural Networks: A Surveycited by 85
  9. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  10. 2023An Outlook into the Future of Egocentric Visioncited by 84
  11. 2023Align and Attend: Multimodal Summarization with Dual Contrastive Lossescited by 65
  12. 2024TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detectioncited by 59
38 more
  1. 2024V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuningcited by 45
  2. 2021Deep Reinforcement Learning in Computer Vision: A Comprehensive Surveycited by 42
  3. 2023VideoXum: Cross-modal Visual and Textural Summarization of Videoscited by 41
  4. 2023MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformercited by 40
  5. 2021Cross-category Video Highlight Detection via Set-based Learningcited by 38
  6. 2022TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliencycited by 35
  7. 2021Reconstructive Sequence-Graph Network for Video Summarizationcited by 34
  8. 2024R2-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Groundingcited by 34
  9. 2023Correlation-Guided Query-Dependency Calibration for Video Temporal Groundingcited by 31
  10. 2025FOCUS: Efficient Keyframe Selection for Long Video Understandingcited by 30
  11. 2023BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videoscited by 29
  12. 2022Toward Unifying Text Segmentation and Long Document Summarizationcited by 28
  13. 2022End-to-end Dense Video Captioning as Sequence Generationcited by 27
  14. 2021Supervised Video Summarization via Multiple Feature Sets with Parallel Attentioncited by 26
  15. 2021GPT2MVS: Generative Pre-trained Transformer-2 for Multi-modal Video Summarizationcited by 26
  16. 2023LvBench: A Benchmark for Long-form Video Understanding with Versatile Multi-modal Question Answeringcited by 25
  17. 2024Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detectioncited by 25
  18. 2024CSTA: CNN-based Spatiotemporal Attention for Video Summarizationcited by 23
  19. 2021Video Summarization through Reinforcement Learning with a 3D Spatio-Temporal U-Netcited by 22
  20. 2022Progressive Video Summarization via Multimodal Self-supervised Learningcited by 21
  21. 2023Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipationcited by 21
  22. 2024AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Descriptioncited by 21
  23. 2021Hierarchical Multimodal Transformer to Summarize Videoscited by 19
  24. 2021IntentVizor: Towards Generic Query Guided Interactive Video Summarizationcited by 18
  25. 2021Learning Triadic Belief Dynamics in Nonverbal Communication from Videoscited by 17
  26. 2024Scaling Up Video Summarization Pretraining with Large Language Modelscited by 17
  27. 2022Learning Pixel-Level Distinctions for Video Highlight Detectioncited by 16
  28. 2025Video Summarization with Large Language Modelscited by 16
  29. 2021UnweaveNet: Unweaving Activity Storiescited by 14
  30. 2021Image-to-Video Generation via 3D Facial Dynamicscited by 13
  31. 2023Query-based Video Summarization with Pseudo Label Supervisioncited by 13
  32. 2023MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videoscited by 12
  33. 2021Video Joint Modelling Based on Hierarchical Transformer for Co-summarizationcited by 10
  34. 2022Continuous Human Action Recognition for Human-Machine Interaction: A Reviewcited by 10
  35. 2022Probing Visual-Audio Representation for Video Highlight Detection via Hard-Pairs Guided Contrastive Learningcited by 9
  36. 2023Learning Segment Similarity and Alignment in Large-Scale Content Based Video Retrievalcited by 9
  37. 2024Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Groundingcited by 9
  38. 2024Towards Automated Movie Trailer Generationcited by 9

50 most cited of 142 All 142 (JSON)