# Collaborative Multi-Mode Pruning for Vision-Language Models

arXiv: https://arxiv.org/abs/2604.02956 · HTML: https://mlsanity.com/paper/2604.02956/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 8 of its 68 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)

## Builds on (most cited first, 8 of 8)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [A Survey of Resource-efficient LLM and Multimodal Foundation Models](https://mlsanity.com/paper/2401.08092/) (2024)
- [A Survey on Transformer Compression](https://mlsanity.com/paper/2402.05964/) (2024)
- [PPT: Token Pruning and Pooling for Efficient Vision Transformers](https://mlsanity.com/paper/2310.01812/) (2023)
- [Multi-Scale And Token Mergence: Make Your ViT More Efficient](https://mlsanity.com/paper/2306.04897/) (2023)
- [ELIP: Efficient Discriminative Language-Image Pre-training with Fewer Vision Tokens](https://mlsanity.com/paper/2309.16738/) (2023)
- [Probe and Skip: Self-Predictive Token Skipping for Efficient Long-Context LLM Inference](https://mlsanity.com/paper/2601.13155/) (2026)

## Built on by (most cited first, 2 of 2)

- [Memory-Efficient Transfer Learning with Fading Side Networks via Masked Dual Path Distillation](https://mlsanity.com/paper/2604.09088/) (2026)
- [PruneForget: Joint Unlearning and Pruning of Vision Models](https://mlsanity.com/paper/2609.32162/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) have advanced rapidly within the unified Transformer architecture, yet their deployment on resource-constrained devices remains challenging due to high computational complexity. While pruning has emerged as an effective technique for compressing VLMs, existing approaches predominantly focus on a single mode by pruning either parameters or tokens, neglecting fully exploring the inherent redundancy in each mode, which leads to substantial performance degradation at high pruning ratios. To address the above limitations, we propose Collaborative Multi-Mode Pruning (CoMP), a novel framework tailored for VLMs by performing joint parameter and token pruning. Specifically, we first design a Collaborative Importance Metric (CIM) that investigates the mutual interference between the coupled parameters and tokens. It incorporates distinct significance of tokens into the computation of parameter importance scores, while simultaneously mitigating the affect of pruned parameters on token importance scores. Moreover, we develop a Multi-Mode Pruning Strategy (MPS) that decomposes the overall pruning process into a sequence of pruning stages, while in each stage we estimate the priory of different pruning modes based on their pruning cost and adaptively shift to the optimal one. Additionally, MPS integrates the historical cost and random exploration, in order to achieve a stable pruning process and avoid local optimum. Extensive experiments across various vision-language tasks and models demonstrate that our method effectively promotes the performance under high pruning ratios by comparing to the state-of-the-art approaches. The source code is available at https://github.com/Wuzimeng/CoMP.git.