mlsanityData to 14 Oct 2026

ERASE: Eliminating Redundant Visual Tokens via Adaptive Two-Stage Token Pruning

arXiv 2605.09982 ↗First posted 11 May 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Recent advances in Vision-Language Models (VLMs) enable large language models (LLMs) to process high-resolution images, significantly improving real-world multimodal understanding. However, this capability introduces a large number of vision tokens, incurring substantial computational overhead. To mitigate this issue, various vision token compression methods have been proposed. Existing methods often estimate different sources of visual redundancy using learned representations or fixed pruning schedules. We propose ERASE, an adaptive two-stage framework that separates image-level redundancy removal from instruction-dependent token pruning. Stage 1 derives image-dependent token retention from lightweight raw-image statistics, while Stage 2 progressively removes instruction-irrelevant tokens across decoder layers. Experiments demonstrate substantial token reduction while preserving accuracy: on Qwen2.5-VL-7B, ERASE retains 95.70% of the original model's accuracy at 25% token retention. Our code is available at https://github.com/Tuna-Luna/ERASE.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 40 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

34 of 40 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity10 Mar
  2. 2026IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruning3 Feb
  3. 2026N"uwa: Mending the Spatial Integrity Torn by VLM Token Pruning3 Feb
  4. 2025Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs12 Jun
  5. 2025VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Models28 May
  6. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models4 Mar
  7. 2025Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More17 Feb
  8. 2025Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Models9 Jan
  9. 2024iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models9 Dec
  10. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  11. 2024Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings29 Nov
  12. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2023Qwen Technical Reportcited by 3,344
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  9. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  10. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  11. 2019Towards VQA Models That Can Readcited by 1,718
  12. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
10 more
  1. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  3. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  4. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  5. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  6. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  7. 2021InfographicVQAcited by 437
  8. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 197
  9. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 161
  10. 2025Vision Function Layer in Multimodal LLMscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization
  2. 2024ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models
  3. 2026Multi-Image Visual Token Pruning in Large Visual Language Models
  4. 2025CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models
  5. 2025GreedyPrune: Retenting Critical Visual Token Set for Large Vision Language Models
  6. 2025Multi-Cue Adaptive Visual Token Pruning for Large Vision-Language Models
  7. 2026Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models
  8. 2026STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models