# GleanVID: Complementary Token Selection for Efficient Video Large Language Models

arXiv: https://arxiv.org/abs/2609.37042 · HTML: https://mlsanity.com/paper/2609.37042/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving.
- Builds on 31 of its 32 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Placed in

- Efficient LLM inference and serving / [Visual token pruning in multimodal models](https://mlsanity.com/inference/visual-token-pruning-in-multimodal-models/)

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models](https://mlsanity.com/paper/2403.06764/) (2024)
- [LLaVA-Video: Video Instruction Tuning With Synthetic Data](https://mlsanity.com/paper/2410.02713/) (2024)
- [VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding](https://mlsanity.com/paper/2501.13106/) (2025)
- [SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference](https://mlsanity.com/paper/2410.04417/) (2024)
- [LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models](https://mlsanity.com/paper/2407.12772/) (2024)
- [VisionZip: Longer is Better but Not Necessary in Vision Language Models](https://mlsanity.com/paper/2412.04467/) (2024)
- [LongVILA: Scaling Long-Context Visual Language Models for Long Videos](https://mlsanity.com/paper/2408.10188/) (2024)
- [MLVU: Benchmarking Multi-task Long Video Understanding](https://mlsanity.com/paper/2406.04264/) (2024)
- [InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling](https://mlsanity.com/paper/2501.12386/) (2025)
- [DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models](https://mlsanity.com/paper/2411.15024/) (2024)
- [PruneVid: Visual Token Pruning for Efficient Video Large Language Models](https://mlsanity.com/paper/2412.16117/) (2024)
- [HoliTom: Holistic Token Merging for Fast Video Large Language Models](https://mlsanity.com/paper/2505.21334/) (2025)
- [FrameFusion: Combining Similarity and Importance for Video Token Reduction on Large Vision Language Models](https://mlsanity.com/paper/2501.01986/) (2024)
- [FastVID: Dynamic Density Pruning for Fast Video Large Language Models](https://mlsanity.com/paper/2503.11187/) (2025)
- [Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Models](https://mlsanity.com/paper/2505.14454/) (2025)
- [FlashVID: Efficient Video Large Language Models via Training-free Tree-based Spatiotemporal Token Merging](https://mlsanity.com/paper/2602.08024/) (2026)
- [TimeExpert: An Expert-Guided Video LLM for Video Temporal Grounding](https://mlsanity.com/paper/2508.01699/) (2025)
- [MMG-Vid: Maximizing Marginal Gains at Segment-level and Token-level for Efficient Video LLMs](https://mlsanity.com/paper/2508.21044/) (2025)

## Abstract (arXiv)

Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the large number of visual tokens. Existing VideoLLM token compression methods largely rely on selection-independent scoring, overlooking cross-frame complementarity and consequently retaining redundant evidence across frames. Instead, we view video token selection as a progressive evidence accumulation process. It aims to retain visual evidence that is individually informative and collectively complementary under a limited token budget. Building on this insight, we introduce GleanVID, a training-free inference acceleration framework for VideoLLMs. Specifically, GleanVID first allocates the global token budget across frames according to temporal novelty and then selects tokens by jointly considering local representativeness and subspace complementarity, thereby preserving richer and less redundant visual evidence. Extensive experiments across diverse VideoLLMs and benchmarks demonstrate that GleanVID consistently achieves state-of-the-art performance. Notably, with only 25% of visual tokens, GleanVID preserves 98.6% of Qwen3-VL's original performance while reducing its prefill latency by 44.7%. On LLaVA-OV-7B, GleanVID at a 25% retention ratio even slightly surpasses the original model.