mlsanityData to 7 Oct 2026

MULTIFLOW: Shifting Towards Task-Agnostic Vision-Language Pruning

arXiv 2404.05621 ↗First posted 8 Apr 2024cs.CVcited by 13

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

While excellent in transfer learning, Vision-Language models (VLMs) come with high computational costs due to their large number of parameters. To address this issue, removing parameters via model pruning is a viable solution. However, existing techniques for VLMs are task-specific, and thus require pruning the network from scratch for each new task of interest. In this work, we explore a new direction: Task-Agnostic Vision-Language Pruning (TA-VLP). Given a pretrained VLM, the goal is to find a unique pruned counterpart transferable to multiple unknown downstream tasks. In this challenging setting, the transferable representations already encoded in the pretrained model are a key aspect to preserve. Thus, we propose Multimodal Flow Pruning (MULTIFLOW), a first, gradient-free, pruning framework for TA-VLP where: (i) the importance of a parameter is expressed in terms of its magnitude and its information flow, by incorporating the saliency of the neurons it connects; and (ii) pruning is driven by the emergent (multimodal) distribution of the VLM parameters after pretraining. We benchmark eight state-of-the-art pruning algorithms in the context of TA-VLP, experimenting with two VLMs, three vision-language tasks, and three pruning ratios. Our experimental results show that MULTIFLOW outperforms recent sophisticated, combinatorial competitors in the vast majority of the cases, paving the way towards addressing TA-VLP. The code is publicly available at https://github.com/FarinaMatteo/multiflow.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 59 of its 75 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

59 of 75 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning14 Oct

Other references · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  10. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  11. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  12. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
38 more
  1. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  2. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  3. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  4. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  5. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  6. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  7. 2019Are Sixteen Heads Really Better than One?cited by 803
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  9. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  10. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  11. 2019Importance Estimation for Neural Network Pruningcited by 531
  12. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  13. 2019The State of Sparsity in Deep Neural Networkscited by 471
  14. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  15. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 398
  16. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  17. 2021Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Conceptscited by 255
  18. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
  19. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  20. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  21. 2023Any-to-Any Generation via Composable Diffusioncited by 203
  22. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 184
  23. 2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 181
  24. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  25. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  26. 2020Sparse GPU Kernels for Deep Learningcited by 138
  27. 2021Patch Slimming for Efficient Vision Transformerscited by 131
  28. 2020Layer-adaptive sparsity for the Magnitude-based Pruningcited by 122
  29. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115
  30. 2020The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Modelscited by 91
  31. 2021Vision Transformer Pruningcited by 77
  32. 2021Global Vision Transformer Pruning with Hessian-Aware Saliencycited by 69
  33. 2021Prune Once for All: Sparse Pre-Trained Language Modelscited by 66
  34. 2022Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Spacecited by 66
  35. 2020Progressive Skeletonization: Trimming more fat from a network at initializationcited by 64
  36. 2023UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformerscited by 51
  37. 2021Playing Lottery Tickets with Vision and Languagecited by 45
  38. 2023The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Mattercited by 42

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 6.

13 papers.

Sort:

By area:

  1. 20252SSP: A Two-Stage Framework for Structured Pruning of LLMscited by 12
  2. 2025Optimal Brain Connection: Towards Efficient Structural Pruningcited by 3
  3. 2024Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Trainingcited by 2
  4. 2025MRC-DETR: An Adaptive Multi-Residual Coupled Transformer for Bare Board PCB Defect Detectioncited by 2
  5. 2026MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMscited by 2
  6. 2026Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiencycited by 2
  7. 2025Safe Vision-Language Models via Unsafe Weights Manipulationcited by 1
  8. 2025Investigating Structural Pruning and Recovery Techniques for Compressing Multimodal Large Language Models: An Empirical Studycited by 1
  9. 2025Towards Efficient VLMs: Information-Theoretic Driven Compression via Adaptive Structural Pruningcited by 1
  10. 2026Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Modelscited by 1
  11. 2026LLaVA-CKD: Bottom-Up Cascaded Knowledge Distillation for Vision-Language Modelscited by 0
  12. 2026Toward Low-Latency Vision-Language Models with Doubly-Correct Predictions in Egocentric Visual Understandingcited by 0
1 more
  1. 2026Are the High-weight Neurons the Important Ones in Image Classification Neural Networks?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026HiPP-Prune: Hierarchical Preference-Conditioned Structured Pruning for Vision-Language Models
  2. 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization
  3. 2026SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models
  4. 2025ZOO-Prune: Training-Free Token Pruning via Zeroth-Order Gradient Estimation in Vision-Language Models
  5. 2026ViTCoP: Accelerating Large Vision-Language Models via Visual and Textual Semantic Collaborative Pruning
  6. 2024Rethinking Pruning for Vision-Language Models: Strategies for Effective Sparsity and Performance Restoration
  7. 2025PLPHP: Per-Layer Per-Head Vision Token Pruning for Efficient Large Vision-Language Models
  8. 2026LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models