mlsanityData to 14 Oct 2026

Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior

arXiv 2512.06866 ↗First posted 7 Dec 2025cs.CV cs.AI cs.CLcited by 15

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Recent advances in Video Large Language Models (VLLMs) have achieved remarkable video understanding capabilities, yet face critical efficiency bottlenecks due to quadratic computational growth with lengthy visual token sequences of long videos. While existing keyframe sampling methods can improve temporal modeling efficiency, additional computational cost is introduced before feature encoding, and the binary frame selection paradigm is found suboptimal. Therefore, in this work, we propose Dynamic Token compression via LLM-guided Keyframe prior (DyToK), a training-free paradigm that enables dynamic token compression by harnessing VLLMs' inherent attention mechanisms. Our analysis reveals that VLLM attention layers naturally encoding query-conditioned keyframe priors, by which DyToK dynamically adjusts per-frame token retention ratios, prioritizing semantically rich frames while suppressing redundancies. Extensive experiments demonstrate that DyToK achieves state-of-the-art efficiency-accuracy tradeoffs. DyToK shows plug-and-play compatibility with existing compression methods, such as VisionZip and FastV, attaining 4.3x faster inference while preserving accuracy across multiple VLLMs, such as LLaVA-OneVision and Qwen2.5-VL. Code is available at https://github.com/yu-lin-li/DyToK .

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 67 of its 75 references that are in the map. First posted on arXiv 7 Dec 2025.

Builds on

67 of 75 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025QuoTA: Query-oriented Token Assignment via CoT Query Decouple for Long Video Comprehension11 Mar
  2. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models4 Mar
  3. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More17 Feb
  4. 2024PruneVid: Visual Token Pruning for Efficient Video Large Language Models20 Dec
  5. 2024Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLM12 Dec
  6. 2024A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs4 Dec
  7. 2024freePruner: A Training-free Approach for Large Multimodal Model Acceleration23 Nov
  8. 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models22 Nov
  9. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  10. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 57

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  10. 2021ViViT: A Video Vision Transformercited by 1,620
  11. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  12. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
38 more
  1. 2021Video Swin Transformercited by 1,091
  2. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  3. 2023Vision Transformers Need Registerscited by 871
  4. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  5. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  6. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  7. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  8. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  9. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
  10. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  11. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  12. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  13. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
  14. 2020Prior Guided Feature Enrichment Network for Few-Shot Segmentationcited by 271
  15. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  16. 2025Adaptive Keyframe Sampling for Long Video Understandingcited by 167
  17. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 166
  18. 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 147
  19. 2024HourVideo: 1-Hour Video-Language Understandingcited by 113
  20. 2021Semi-supervised Semantic Segmentation with Directional Context-aware Consistencycited by 112
  21. 2021ResLT: Residual Learning for Long-tailed Recognitioncited by 96
  22. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  23. 2025M-LLM Based Video Frame Selection for Efficient Video Understandingcited by 88
  24. 2021Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentationcited by 87
  25. 2023Hierarchical Dense Correlation Distillation for Few-Shot Segmentationcited by 84
  26. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 83
  27. 2024Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentationcited by 61
  28. 2020Generalized Few-shot Semantic Segmentationcited by 59
  29. 2024KeyVideoLLM: Towards Large-scale Video Keyframe Selectioncited by 54
  30. 2022Generalized Parametric Contrastive Learningcited by 45
  31. 2024OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentationcited by 44
  32. 2025DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perceptioncited by 33
  33. 2021PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Maskcited by 31
  34. 2024Unified Language-driven Zero-shot Domain Adaptationcited by 30
  35. 2025FOLDER: Accelerating Multi-modal Large Language Models with Enhanced Performancecited by 30
  36. 2024Scalable Language Model with Generalized Continual Learningcited by 28
  37. 2025Mitigating Object Hallucinations via Sentence-Level Early Interventioncited by 28
  38. 2024GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understandingcited by 26

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 15.

15 papers.

Sort:

By area:

  1. 2026FlashVID: Efficient Video Large Language Models via Training-free Tree-based Spatiotemporal Token Mergingcited by 18
  2. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Evictioncited by 11
  3. 2026Unified Spatiotemporal Token Compression for Video-LLMs at Ultra-Low Retentioncited by 6
  4. 2026Efficient Reasoning with Balanced Thinkingcited by 4
  5. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4
  6. 2026Towards a Dynamic and Fixed-budget Memory Bank for Efficient Streaming Video Understandingcited by 4
  7. 2026LDDR: Linear-DPP-Based Dynamic-Resolution Frame Sampling for Video MLLMscited by 2
  8. 2026KiToke: Kernel-based Interval-aware Token Compression for Video Large Language Modelscited by 1
  9. 2026MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cuescited by 1
  10. 2026When and Where to Look: Adaptive Visual Evidence Scheduling for Efficient Long Video Understandingcited by 1
  11. 2026VideoRouter: Query-Adaptive Dual Routing for Efficient Long-Video Understandingcited by 0
  12. 2026CLIP Tricks You: Training-free Token Pruning for Efficient Pixel Grounding in Large VIsion-Language Modelscited by 0
3 more
  1. 2026CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Modelscited by 0
  2. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  3. 2026Video Evidence Indexing: Learning Where to Look from Video Previews for Token-Budgeted Long-Video Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Video Codec-Guided Training-Free Token Compression for Video Large Language Models
  2. 2026DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMs
  3. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
  4. 2026TTF: Temporal Token Fusion for Efficient Video-Language Model
  5. 2026Rethinking Visual Token Compression for Video Large Language Models: A Simple Yet Strong Baseline
  6. 2026Rethinking Video Token Compression with a Global Codebook: Learning Once, Compressing Everywhere
  7. 2026Token Reduction via Local and Global Contexts Optimization for Efficient Video Large Language Models
  8. 2024TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models