# Large Model based Sequential Keyframe Extraction for Video Summarization

arXiv: https://arxiv.org/abs/2401.04962 · HTML: https://mlsanity.com/paper/2401.04962/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 24 references that are in the map.
- First posted on arXiv 10 Jan 2024.

## Builds on (most cited first, 4 of 4)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [TransNet V2: An effective deep network architecture for fast shot transition detection](https://mlsanity.com/paper/2008.04838/) (2020)
- [Deep Unsupervised Key Frame Extraction for Efficient Video Classification](https://mlsanity.com/paper/2211.06742/) (2022)

## Built on by (most cited first, 9 of 9)

- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [FrameOracle: Learning What to See and How Much to See in Videos](https://mlsanity.com/paper/2510.03584/) (2025)
- [HierSum: A Global and Local Attention Mechanism for Video Summarization](https://mlsanity.com/paper/2504.18689/) (2025)
- [TriPSS: A Tri-Modal Keyframe Extraction Framework Using Perceptual, Structural, and Semantic Representations](https://mlsanity.com/paper/2506.05395/) (2025)
- [From Captions to Keyframes: KeyScore for Multimodal Frame Scoring and Video-Language Understanding](https://mlsanity.com/paper/2510.06509/) (2025)
- [Causal-Aware Intelligent QoE Optimization for VR Interaction with Adaptive Keyframe Extraction](https://mlsanity.com/paper/2506.19890/) (2025)
- [Re-Identifying K=ak=a with AI-Automated Video Key Frame Extraction](https://mlsanity.com/paper/2510.08775/) (2025)
- [Fusionista2.0: Efficiency Retrieval System for Large-Scale Datasets](https://mlsanity.com/paper/2511.12255/) (2025)
- [Multilingual, Multimodal Pipeline for Creating Authentic and Structured Fact-Checked Claim Dataset](https://mlsanity.com/paper/2601.07985/) (2026)

## Abstract (arXiv)

Keyframe extraction aims to sum up a video's semantics with the minimum number of its frames. This paper puts forward a Large Model based Sequential Keyframe Extraction for video summarization, dubbed LMSKE, which contains three stages as below. First, we use the large model "TransNetV21" to cut the video into consecutive shots, and employ the large model "CLIP2" to generate each frame's visual feature within each shot; Second, we develop an adaptive clustering algorithm to yield candidate keyframes for each shot, with each candidate keyframe locating nearest to a cluster center; Third, we further reduce the above candidate keyframes via redundancy elimination within each shot, and finally concatenate them in accordance with the sequence of shots as the final sequential keyframes. To evaluate LMSKE, we curate a benchmark dataset and conduct rich experiments, whose results exhibit that LMSKE performs much better than quite a few SOTA competitors with average F1 of 0.5311, average fidelity of 0.8141, and average compression ratio of 0.9922.