mlsanityData to 14 Oct 2026

Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiency

arXiv 2604.24380 ↗First posted 27 Apr 2026cs.CLcited by 3

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

While Large Vision Language Models (LVLMs) demonstrate impressive capabilities, their substantial computational and memory requirements pose deployment challenges on resource-constrained edge devices. Current parameter reduction techniques primarily involve training LVLMs from small language models, but these methods offer limited flexibility and remain computationally intensive. We study a complementary route: compressing existing LVLMs by applying structured pruning to the language model backbone, followed by lightweight recovery training. Specifically, we investigate two structural pruning paradigms: layerwise and widthwise pruning, and pair them with supervised finetuning and knowledge distillation on logits and hidden states. Additionally, we assess the feasibility of conducting recovery training with only a small fraction of the available data. Our results show that widthwise pruning generally maintains better performance in low-resource scenarios, where computational resources are limited or there is insufficient finetuning data. As for the recovery training, finetuning only the multimodal projector is sufficient at small compression levels. Furthermore, a combination of supervised finetuning and hidden-state distillation yields optimal recovery across various pruning levels. Notably, effective recovery can be achieved using just 5% of the original data, while retaining over 95% of the original performance. Through empirical study on three representative LVLM families ranging from 3B to 7B parameters, this study offers actionable insights for practitioners to compress LVLMs without extensive computation resources or sufficient data. The code base is available at https://github.com/YiranHuangIrene/VLMCompression.git.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 64 of its 71 references that are in the map. First posted on arXiv 27 Apr 2026.

Builds on

64 of 71 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025AutoPrune: Each Complexity Deserves a Pruning Policy28 Sep
  2. 2025CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models24 Aug
  3. 2025Investigating Structural Pruning and Recovery Techniques for Compressing Multimodal Large Language Models: An Empirical Study28 Jul
  4. 2024MULTIFLOW: Shifting Towards Task-Agnostic Vision-Language Pruning8 Apr
  5. 2024LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models22 Mar

Other references · 59

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  10. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  11. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  12. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
38 more
  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  2. 2020Knowledge Distillation: A Surveycited by 1,603
  3. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  4. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  5. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  6. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  7. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  8. 2023A Survey on Multimodal Large Language Modelscited by 909
  9. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  10. 2019Are Sixteen Heads Really Better than One?cited by 816
  11. 2016A Diagram Is Worth A Dozen Imagescited by 804
  12. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  13. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  14. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  15. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
  16. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 508
  17. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
  18. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
  19. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 403
  20. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  21. 2024Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Modelscited by 339
  22. 2016Benefits of depth in neural networkscited by 250
  23. 2020MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformerscited by 245
  24. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  25. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
  26. 2023DepGraph: Towards Any Structural Pruningcited by 203
  27. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
  28. 2024Compact Language Models via Pruning and Knowledge Distillationcited by 148
  29. 2021Group Fisher Pruning for Practical Network Compressioncited by 131
  30. 2024LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Modelcited by 129
  31. 2024Efficient Multimodal Learning from Data-centric Perspectivecited by 122
  32. 2019Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networkscited by 118
  33. 2020On the Effect of Dropping Layers of Pre-trained Transformer Modelscited by 118
  34. 2024SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blockscited by 97
  35. 2019A Signal Propagation Perspective for Pruning Neural Networks at Initializationcited by 96
  36. 2019Are All Layers Created Equal?cited by 94
  37. 2023CLIP-KD: An Empirical Study of CLIP Model Distillationcited by 87
  38. 2024LLM Pruning and Distillation in Practice: The Minitron Approachcited by 82

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026StepPrune: Adaptive Sequential Visual Token Selection across Multimodal Large Language Modelscited by 0
  2. 2026Recovering Aggressively Pruned Vision-Language-Action Models with Offline Hidden-State Distillationcited by 0
  3. 2026Retraining-Free Depth Pruning of Vision Transformers: A Cross Domain Study of Layer Redundancycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models
  2. 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization
  3. 2024Pruning All-Rounder: Rethinking and Improving Inference Efficiency for Large Vision Language Models
  4. 2026Topology-Aware Layer Pruning for Large Vision-Language Models
  5. 2024Rethinking Pruning for Vision-Language Models: Strategies for Effective Sparsity and Performance Restoration
  6. 2025EfficientLLaVA:Generalizable Auto-Pruning for Large Vision-language Models
  7. 2026SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models
  8. 2026Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models