mlsanityData to 7 Oct 2026

ETC: Extreme Token Compression via Task-aware Visual Information Distillation in VLMs

arXiv 2606.00543 ↗First posted 30 May 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

In Vision-Language Models (VLMs), high-resolution images produce a large number of visual tokens, resulting in high computational costs and KV-cache overhead during inference. To address this problem, we propose an Extreme Token Compression (ETC) framework that minimizes task loss when reducing the number of input tokens based on the principle of variational information distillation. Specifically, from an information-theoretic perspective, we show that minimizing task loss requires the compact representation to preserve the instruction-aware sufficient statistic of the task-relevant visual information for prediction. In practice, ETC leverages text-to-image cross-attention to weight the original visual features to approximate the latent instruction-aware predictive statistic. Moreover, ETC introduces a variational information distillation, enabling the compact representation to preserve the essential information to recover this predictive statistic. Experiments on LLaVA-1.5-7B and Qwen3-VL-2B show that ETC remains effective even under single-token compression, substantially reducing KV-cache overhead while retaining strong task performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 45 references that are in the map. First posted on arXiv 30 May 2026.

Builds on

39 of 45 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025PACT: Pruning and Clustering-Based Token Reduction for Faster Visual Language Models11 Apr
  2. 2025TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model24 Mar
  3. 2024ST3: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming28 Dec
  4. 2024PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models12 Dec
  5. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  6. 2024Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction30 Nov
  7. 2024ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models30 Nov
  8. 2024Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters5 Nov
  9. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
  10. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models16 Sep
  11. 2024TokenPacker: Efficient Visual Projector for Multimodal LLM2 Jul
  12. 2024VoCo-LLaMA: Towards Vision Compression with Large Language Models18 Jun
3 more
  1. 2024Matryoshka Query Transformer for Large Vision-Language Models29 May
  2. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  3. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  9. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  10. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  11. 2019Towards VQA Models That Can Readcited by 1,696
  12. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
12 more
  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  2. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  3. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  4. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  5. 2019Variational Information Distillation for Knowledge Transfercited by 321
  6. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  7. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compressioncited by 41
  8. 2024TempMe: Video Temporal Token Merging for Efficient Text-Video Retrievalcited by 39
  9. 2024SCOPE: Optimizing Key-Value Cache Compression in Long-context Generationcited by 20
  10. 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understandingcited by 17
  11. 2024Efficient Visual Transformer by Learnable Token Mergingcited by 7
  12. 2024Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queriescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMs
  2. 2026VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression
  3. 2025Hybrid Token Compression for Vision-Language Models
  4. 2025Variation-aware Vision Token Dropping for Faster Large Vision-Language Models
  5. 2026Beyond Selection: Token Parameterization for Extreme Visual Token Compression
  6. 2024VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration
  7. 2026DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference
  8. 2025FCoT-VL:Advancing Text-oriented Large Vision-Language Models with Efficient Visual Token Compression