mlsanityData to 14 Oct 2026

FastVID: Dynamic Density Pruning for Fast Video Large Language Models

arXiv 2503.11187 ↗First posted 14 Mar 2025cs.CVcited by 52

Not on the map. Why?

Cited by 28 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Video Large Language Models have demonstrated strong video understanding capabilities, yet their practical deployment is hindered by substantial inference costs caused by redundant video tokens. Existing pruning techniques fail to effectively exploit the spatiotemporal redundancy present in video data. To bridge this gap, we perform a systematic analysis of video redundancy from two perspectives: temporal context and visual context. Leveraging these insights, we propose Dynamic Density Pruning for Fast Video LLMs termed FastVID. Specifically, FastVID dynamically partitions videos into temporally ordered segments to preserve temporal structure and applies a density-based token pruning strategy to maintain essential spatial and temporal information. Our method significantly reduces computational overhead while maintaining temporal and visual integrity. Extensive evaluations show that FastVID achieves state-of-the-art performance across various short- and long-video benchmarks on leading Video LLMs, including LLaVA-OneVision, LLaVA-Video, Qwen2-VL, and Qwen2.5-VL. Notably, on LLaVA-OneVision-7B, FastVID effectively prunes 90.3% of video tokens, reduces FLOPs to 8.3%, and accelerates the LLM prefill stage by 7.1×, while maintaining 98.0% of the original accuracy. The code is available at https://github.com/LunarShen/FastVID.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 49 of its 61 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

49 of 61 references.

References · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  11. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
37 more
  1. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  3. 2024Qwen2 Technical Reportcited by 2,120
  4. 2020GLU Variants Improve Transformercited by 1,804
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  7. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  9. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  10. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  11. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  12. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  13. 2022Token Merging: Your ViT But Fastercited by 1,021
  14. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  15. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  16. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  17. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  18. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  19. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  20. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Modelscited by 402
  21. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  22. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  23. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 356
  24. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  25. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
  26. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  27. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 294
  28. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inferencecited by 139
  29. 2024DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Modelscited by 96
  30. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  31. 2024Apollo: An Exploration of Video Understanding in Large Multimodal Modelscited by 89
  32. 2024PruneVid: Visual Token Pruning for Efficient Video Large Language Modelscited by 67
  33. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 64
  34. 2024FrameFusion: Combining Similarity and Importance for Video Token Reduction on Large Vision Language Modelscited by 54
  35. 2024TempMe: Video Temporal Token Merging for Efficient Text-Video Retrievalcited by 40
  36. 2024[CLS] Token Tells Everything Needed for Training-free Efficient MLLMscited by 26
  37. 2024ST3: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimmingcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 40.

52 papers.

Sort:

By area:

  1. 2025HoliTom: Holistic Token Merging for Fast Video Large Language Modelscited by 58
  2. 2025StreamingTOM: Streaming Token Compression for Efficient Video Understandingcited by 43
  3. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmarkcited by 37
  4. 2025LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMscited by 26
  5. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 25
  6. 2026LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMscited by 24
  7. 2025OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Modelscited by 22
  8. 2025Bidirectional Sparse Attention for Faster Video Diffusion Trainingcited by 19
  9. 2025Plug-and-Play 1.x-Bit KV Cache Quantization for Video Large Language Modelscited by 13
  10. 2026ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inferencecited by 9
  11. 2026Attend Before Attention: Efficient and Scalable Video Understanding via Autoregressive Gazingcited by 9
  12. 2026ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoningcited by 8
38 more
  1. 2025UniComp: Rethinking Video Compression Through Informational Uniquenesscited by 7
  2. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 7
  3. 2026DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compressioncited by 6
  4. 2026Token Reduction via Local and Global Contexts Optimization for Efficient Video Large Language Modelscited by 5
  5. 2026EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMscited by 5
  6. 2025Focus: A Streaming Concentration Architecture for Efficient Vision-Language Modelscited by 3
  7. 2026Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Groundingcited by 3
  8. 2026OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Modelscited by 3
  9. 2026StreamScout: Learning When to Look Deeper for Streaming Video Understandingcited by 3
  10. 2026Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unificationcited by 2
  11. 2026V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Modelscited by 2
  12. 2026LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMscited by 2
  13. 2026DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMscited by 2
  14. 2026Swift Sampling: Selecting Temporal Surprises via Taylor Seriescited by 2
  15. 2026ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMscited by 2
  16. 2026WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensationcited by 2
  17. 2025VideoScaffold: Elastic-Scale Visual Hierarchies for Streaming Video Understanding in MLLMscited by 1
  18. 2026Dynamic Token Compression for Efficient Video Understanding through Reinforcement Learningcited by 1
  19. 2026KiToke: Kernel-based Interval-aware Token Compression for Video Large Language Modelscited by 1
  20. 2026Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understandingcited by 1
  21. 2026EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMscited by 1
  22. 2026OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Modelscited by 1
  23. 2026CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transportcited by 1
  24. 2026Rethinking Long-Video Efficiency: A Joint Allocation Perspective on Frames, Pixels, and Front-End Latencycited by 1
  25. 2025ShaRP: SHAllow-LayeR Pruning for Efficient Video Large Language Modelscited by 0
  26. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategiescited by 0
  27. 2026Tango: Taming Visual Signals for Efficient Video Large Language Modelscited by 0
  28. 2026POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMscited by 0
  29. 2026Sink-Token-Aware Pruning for Fine-Grained Video Understanding in Efficient Video LLMscited by 0
  30. 2026VLMaxxing through FrameMogging Training-Free Anti-Recomputation for Video Vision-Language Modelscited by 0
  31. 2026FastOCR: Dynamic Visual Fixation via KV Cache Pruning for Efficient Document Parsingcited by 0
  32. 2026GeoTrace: Geometry-Aware Trajectory Token Compression for Video Large Language Modelscited by 0
  33. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0
  34. 2026From Token Importance to Conditional Removability: Rethinking Visual Token Pruning in Multimodal Large Language Modelscited by 0
  35. 2026KeyRec: Bounded Visual Memory for Streaming and Long-Video Understandingcited by 0
  36. 2026GleanVID: Complementary Token Selection for Efficient Video Large Language Modelscited by 0
  37. 2026FlashGaze: Training-Free Multi-Scale Patch Pruning For Efficient Video Understandingcited by 0
  38. 2026Video Codec-Guided Training-Free Token Compression for Video Large Language Modelscited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026HieraVid: Hierarchical Token Pruning for Fast Video Large Language Models
  2. 2026GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models
  3. 2026FlashVID: Efficient Video Large Language Models via Training-free Tree-based Spatiotemporal Token Merging
  4. 2025MMG-Vid: Maximizing Marginal Gains at Segment-level and Token-level for Efficient Video LLMs
  5. 2025StreamingAssistant: Efficient Visual Token Pruning for Accelerating Online Video Understanding
  6. 2025Language-Guided Temporal Token Pruning for Efficient VideoLLM Processing
  7. 2025Efficient Video Sampling: Pruning Temporally Redundant Tokens for Faster VLM Inference
  8. 2025AdaTP: Attention-Debiased Token Pruning for Video Large Language Models