mlsanityData to 7 Oct 2026

KeyVideoLLM: Towards Large-scale Video Keyframe Selection

arXiv 2407.03104 ↗First posted 3 Jul 2024cs.CV cs.CL cs.MMcited by 52

Not on the map. Why?

Cited by 22 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recently, with the rise of web videos, managing and understanding large-scale video datasets has become increasingly important. Video Large Language Models (VideoLLMs) have emerged in recent years due to their strong video understanding capabilities. However, training and inference processes for VideoLLMs demand vast amounts of data, presenting significant challenges to data management, particularly regarding efficiency, robustness, and effectiveness. In this work, we present KeyVideoLLM, a text-video frame similarity-based keyframe selection method designed to manage VideoLLM data efficiently, robustly, and effectively. Specifically, KeyVideoLLM achieves a remarkable data compression rate of up to 60.9 times, substantially lowering disk space requirements, which proves its high efficiency. Additionally, it maintains a 100% selection success rate across all video formats and scales, enhances processing speed by up to 200 times compared to existing keyframe selection methods, and does not require hyperparameter tuning. Beyond its outstanding efficiency and robustness, KeyVideoLLM further improves model performance in video question-answering tasks during both training and inference stages. Notably, it consistently achieved the state-of-the-art (SoTA) experimental results on diverse datasets.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

24 of 37 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  4. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  5. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  6. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  7. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  8. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  9. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  10. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
  11. 2023Multimodal Large Language Models: A Surveycited by 193
  12. 2022X-Pool: Cross-Modal Language-Video Attention for Text-Video Retrievalcited by 127
12 more
  1. 2023BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questionscited by 125
  2. 2024MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokenscited by 116
  3. 2023MoDS: Model-oriented Data Selection for Instruction Tuningcited by 107
  4. 2021VLM: Task-agnostic Video-Language Model Pre-training for Video Understandingcited by 102
  5. 2021Video Summarization Using Deep Neural Networks: A Surveycited by 85
  6. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  7. 2024Koala: Key frame-conditioned long video-LLMcited by 57
  8. 2023FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placementcited by 52
  9. 2023Rethinking the Instruction Quality: LIFT is What You Needcited by 33
  10. 2024Semantically consistent Video-to-Audio Generation using Multimodal Language Large Modelcited by 11
  11. 2023From BERT to GPT-3 Codex: Harnessing the Potential of Very Large Language Models for Data Managementcited by 10
  12. 2023Lingua Manga: A Generic Large Language Model Centric System for Data Curationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 23; 2026: 26.

52 papers.

Sort:

By area:

  1. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 80
  2. 2025Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMscited by 72
  3. 2025MDP3: A Training-free Approach for List-wise Frame Selection in Video-LLMscited by 40
  4. 2025FOCUS: Efficient Keyframe Selection for Long Video Understandingcited by 30
  5. 2025Improving LLM Video Understanding with 16 Frames Per Secondcited by 26
  6. 2025Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMscited by 20
  7. 2025Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processingcited by 18
  8. 2025TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learningcited by 18
  9. 2025REVISOR: Beyond Textual Reflection, Towards Multimodal Introspective Reasoning in Long-Form Video Understandingcited by 15
  10. 2025Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Priorcited by 15
  11. 2025ViaRL: Adaptive Temporal Grounding via Visual Iterated Amplification Reinforcement Learningcited by 11
  12. 2025Position: Foundation Models Need Digital Twin Representationscited by 10
38 more
  1. 2025AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understandingcited by 9
  2. 2026ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoningcited by 8
  3. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 8
  4. 2025FrameOracle: Learning What to See and How Much to See in Videoscited by 7
  5. 2026AdaCodec: A Predictive Visual Code for Video MLLMscited by 6
  6. 2024BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Modelscited by 5
  7. 2025Language-Guided Temporal Token Pruning for Efficient VideoLLM Processingcited by 4
  8. 2025GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memorycited by 4
  9. 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learningcited by 3
  10. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  11. 2025TriPSS: A Tri-Modal Keyframe Extraction Framework Using Perceptual, Structural, and Semantic Representationscited by 2
  12. 2025Agentic Aerial Cinematography: From Dialogue Cues to Cinematic Trajectoriescited by 2
  13. 2026ViSIL: Unified Evaluation of Information Loss in Multimodal Video Captioningcited by 2
  14. 2026Understanding Multimodal Complementarity for Single-Frame Action Anticipationcited by 2
  15. 2026TiFRe: Text-guided Video Frame Reduction for Efficient Video Multi-modal Large Language Modelscited by 2
  16. 2026LDDR: Linear-DPP-Based Dynamic-Resolution Frame Sampling for Video MLLMscited by 2
  17. 2026PEEK: Picking Essential frames via Efficient Knowledge distillationcited by 2
  18. 2025Mobile-VideoGPT: Fast and Accurate Model for Mobile Video Understandingcited by 1
  19. 2025KFFocus: Highlighting Keyframes for Enhanced Video Understandingcited by 1
  20. 2025AnchorSync: Global Consistency Optimization for Long Video Editingcited by 1
  21. 2025SPIKE-RL: Video-LLMs meet Bayesian Surprisecited by 1
  22. 2025From Captions to Keyframes: KeyScore for Multimodal Frame Scoring and Video-Language Understandingcited by 1
  23. 2026AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understandingcited by 1
  24. 2026GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Groundingcited by 1
  25. 2024Sparrow: Data-Efficient Video-LLM with Text-to-Image Augmentationcited by 0
  26. 2026Improving Video Question Answering through query-based frame selectioncited by 0
  27. 2026STEC: A Reference-Free Spatio-Temporal Entropy Coverage Metric for Evaluating Sampled Video Framescited by 0
  28. 2026Learning Question-Aware Keyframe Selection with Synthetic Supervision for Video Question Answeringcited by 0
  29. 2026Storing Less, Finding More: How Novelty Filtering Improves Cross-Modal Retrieval on Edge Camerascited by 0
  30. 2026TimeProVe: Propose, then Verify for Efficient Long Video Temporal Reasoning in Activities of Daily Livingcited by 0
  31. 2026Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigationcited by 0
  32. 2026Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understandingcited by 0
  33. 2026VisualRouter: Query-Grounded Visual Sampling for Long Video Understandingcited by 0
  34. 2026Evidence-Driven Dynamic Visual Selector for Efficient Long Video Understandingcited by 0
  35. 2026RIDGE: Region-Informed Derivative-Guided Evidence Selection for Long Video Understandingcited by 0
  36. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  37. 2026Evaluation of MLLM-Agnostic Plug-and-Play Keyframe Selection Methods for Long Video Understandingcited by 0
  38. 2026VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understandingcited by 0

50 most cited of 52 All 52 (JSON)