mlsanityData to 14 Oct 2026

Seeing More, Saying More: Lightweight Language Experts are Dynamic Video Token Compressors

arXiv 2509.00969 ↗First posted 31 Aug 2025cs.CVcited by 2

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Recent advancements in large video-language models have revolutionized video understanding tasks. However, their efficiency is significantly constrained by processing high volumes of visual tokens. Existing token compression strategies apply a fixed compression ratio, ignoring the variability in semantic density among different video clips. Consequently, this lead to inadequate representation of information-rich clips due to insufficient tokens and unnecessary computation on static or content-poor ones. To address this, we propose LangDC, a Language-aware Dynamic Token Compressor. LangDC leverages a lightweight language model to describe video clips, converting them into soft caption tokens as visual representations. Trained with our proposed semantic density-aware supervision, LangDC aims to 1) cover key visual cues necessary for downstream task reasoning and 2) dynamically adjust compression ratios based on scene richness, reflected by descriptions length. Our design mimics how humans dynamically express what they see: complex scenes (seeing more) elicit more detailed language to convey nuances (saying more), whereas simpler scenes are described with fewer words. Experimental results show that our method reduces FLOPs by 49% compared to VideoGPT+ while maintaining competitive performance. Furthermore, qualitative results demonstrate our approach adaptively adjusts the token compression ratio based on video segment richness.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 30 of its 47 references that are in the map. First posted on arXiv 31 Aug 2025.

Builds on

30 of 47 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  7. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  8. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  10. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  11. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  12. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
18 more
  1. 2022Token Merging: Your ViT But Fastercited by 1,021
  2. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  3. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  4. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  5. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  6. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  7. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
  8. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understandingcited by 406
  9. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  10. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  11. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 222
  12. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understandingcited by 213
  13. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
  14. 2025LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Tokencited by 166
  15. 2019Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)cited by 128
  16. 2024ST-LLM: Large Language Models Are Effective Temporal Learnerscited by 126
  17. 2024What If We Recaption Billions of Web Images with LLaMA-3?cited by 87
  18. 2024Multi-granularity Correspondence Learning from Long-term Noisy Videoscited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategiescited by 0
  2. 2026VideoRouter: Query-Adaptive Dual Routing for Efficient Long-Video Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
  2. 2026Small Vision-Language Models are Smart Compressors for Long Video Understanding
  3. 2026TTF: Temporal Token Fusion for Efficient Video-Language Model
  4. 2026Rethinking Visual Token Compression for Video Large Language Models: A Simple Yet Strong Baseline
  5. 2025From Frames to Clips: Training-free Adaptive Key Clip Selection for Long-Form Video Understanding
  6. 2026Token Reduction via Local and Global Contexts Optimization for Efficient Video Large Language Models
  7. 2025Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior
  8. 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMs