mlsanityData to 14 Oct 2026

CLASP: Class-Adaptive Layer Fusion and Dual-Stage Pruning for Multimodal Large Language Models

arXiv 2604.12767 ↗First posted 14 Apr 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) suffer from substantial computational overhead due to the high redundancy in visual token sequences. Existing approaches typically address this issue using single-layer Vision Transformer (ViT) features and static pruning strategies. However, such fixed configurations are often brittle under diverse instructions. To overcome these limitations, we propose CLASP, a plug-and-play token reduction framework based on class-adaptive layer fusion and dual-stage pruning. Specifically, CLASP first constructs category-specific visual representations through multi-layer vision feature fusion. It then performs dual-stage pruning, allocating the token budget between attention-salient pivot tokens for relevance and redundancy-aware completion tokens for coverage. Through class-adaptive pruning, CLASP enables prompt-conditioned feature fusion and budget allocation, allowing aggressive yet robust visual token reduction. Extensive experiments demonstrate that CLASP consistently outperforms existing methods across a wide range of benchmarks, pruning ratios, and MLLM architectures. Code will be available at https://github.com/Yunkaidang/CLASP.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 55 references that are in the map. First posted on arXiv 14 Apr 2026.

Builds on

45 of 55 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026N"uwa: Mending the Spatial Integrity Torn by VLM Token Pruning3 Feb
  2. 2025Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention3 Oct
  3. 2025Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs12 Jun
  4. 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization28 May
  5. 2025DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models4 Mar
  6. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?17 Feb
  7. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Models5 Dec
  8. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs2 Dec
  9. 2024Filter, Correlate, Compress: Training-Free Token Reduction for MLLM Acceleration26 Nov
  10. 2024Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model16 Nov
  11. 2024PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction22 Oct
  12. 2024SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference6 Oct
3 more
  1. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference9 May
  2. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar
  3. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models11 Mar

Other references · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  11. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  12. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
18 more
  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  2. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  3. 2019Towards VQA Models That Can Readcited by 1,718
  4. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  5. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  7. 2022Token Merging: Your ViT But Fastercited by 1,021
  8. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  9. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  10. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 244
  11. 2023Multimodal Large Language Models: A Surveycited by 195
  12. 2016TGIF: A New Dataset and Benchmark on Animated GIF Descriptioncited by 124
  13. 2024Efficient Multimodal Large Language Models: A Surveycited by 96
  14. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 50
  15. 2024Exploring Token Pruning in Vision State Space Modelscited by 28
  16. 2025Multi-Layer Visual Feature Fusion in Multimodal LLMs: Methods, Analysis, and Best Practicescited by 22
  17. 2025FUSE-RSVLM: Feature Fusion Vision-Language Model for Remote Sensingcited by 3
  18. 2026HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Frameworkcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models
  2. 2026SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Models
  3. 2026SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models
  4. 2026Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models
  5. 2026EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMs
  6. 2025CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models
  7. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models
  8. 2025TrimTokenator: Towards Adaptive Visual Token Pruning for Large Multimodal Models