mlsanityData to 14 Oct 2026

Just Respect the Semantics: Image-Token Compression via Pre-LLM Semantic Variable-Resolution Tokenization and In-LLM Relevance Pruning

arXiv 2610.18690 ↗First posted 13 Oct 2026cs.CV cs.AI cs.LGcited by 0

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Image-token compression must reduce computation without hurting the semantics. We introduce Varest, guided by semantic fungibility: tokens within a coherent semantic region can be summarized coarsely, while object boundaries and literal text require finer distinctions. Object and text localizers guide variable-resolution merging, preserving spatial coverage and finer detail at boundaries and text. Learned [REL] registers then prune visual tokens after early decoder processing. Training across budgets enables one checkpoint per backbone. Across three backbones and 13 benchmarks, Varest outperforms 15 baselines in mean performance retention at every evaluated budget, retaining 86.10-92.33% of full-token performance with only 5.56% of the original layer-averaged image-token count.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 32 of its 36 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

32 of 36 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models27 Apr
  2. 2026HiDrop: Hierarchical Vision Token Reduction in MLLMs via Late Injection, Concave Pyramid Pruning, and Early Exit27 Feb
  3. 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMs23 Feb
  4. 2026OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport22 Feb
  5. 2026DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference21 Feb
  6. 2026N"uwa: Mending the Spatial Integrity Torn by VLM Token Pruning3 Feb
  7. 2025When Token Pruning is Worse than Random: Understanding Visual Token Information in VLLMs8 Dec
  8. 2025CORE: Compact Object-centric REpresentations as a New Paradigm for Token Merging in LVLMs18 Nov
  9. 2025VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization7 Aug
  10. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models4 Mar
  11. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 21

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  7. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  8. 2019Towards VQA Models That Can Readcited by 1,718
  9. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  10. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  11. 2016Modeling Context in Referring Expressionscited by 1,063
  12. 2022Token Merging: Your ViT But Fastercited by 1,021
9 more
  1. 2019Attention is not Explanationcited by 929
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  3. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  4. 2016A Diagram Is Worth A Dozen Imagescited by 804
  5. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  6. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inferencecited by 391
  7. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 313
  8. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reductioncited by 294
  9. 2023Teaching CLIP to Count to Tencited by 198

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Selection: Token Parameterization for Extreme Visual Token Compression
  2. 2024FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compression
  3. 2025AdaTok: Adaptive Token Compression with Object-Aware Representations for Efficient Multimodal LLMs
  4. 2026Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity
  5. 2024Visual-Word Tokenizer: Beyond Fixed Sets of Tokens in Vision Transformers
  6. 2025Hybrid Token Compression for Vision-Language Models
  7. 2026Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning
  8. 2026PixelPrune: Pixel-Level Adaptive Visual Token Reduction via Predictive Coding