mlsanityData to 7 Oct 2026

DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation

arXiv 2004.02164 ↗First posted 5 Apr 2020cs.CV cs.LG stat.MLcited by 47

Not on the map. Why?

Abstract (arXiv)

Budgeted pruning is the problem of pruning under resource constraints. In budgeted pruning, how to distribute the resources across layers (i.e., sparsity allocation) is the key problem. Traditional methods solve it by discretely searching for the layer-wise pruning ratios, which lacks efficiency. In this paper, we propose Differentiable Sparsity Allocation (DSA), an efficient end-to-end budgeted pruning flow. Utilizing a novel differentiable pruning process, DSA finds the layer-wise pruning ratios with gradient-based optimization. It allocates sparsity in continuous space, which is more efficient than methods based on discrete evaluation and search. Furthermore, DSA could work in a pruning-from-scratch manner, whereas traditional budgeted pruning methods are applied to pre-trained models. Experimental results on CIFAR-10 and ImageNet show that DSA could achieve superior performance than current iterative budgeted pruning methods, and shorten the time cost of the overall pruning process by at least 1.5x in the meantime.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 14; 2022: 8; 2023: 9; 2024: 9; 2025: 7.

47 papers.

Sort:

By area:

  1. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  2. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  3. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  4. 2021Towards Efficient Tensor Decomposition-Based DNN Model Compression with Optimization Frameworkcited by 52
  5. 2024Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costscited by 51
  6. 2021Manifold Regularized Dynamic Network Pruningcited by 43
  7. 2022CHEX: CHannel EXploration for CNN Model Compressioncited by 36
  8. 2022Learning Best Combination for Efficient N:M Sparsitycited by 32
  9. 2021Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learningcited by 30
  10. 2021Carrying out CNN Channel Pruning in a White Boxcited by 22
  11. 2024Pruning for Robust Concept Erasing in Diffusion Modelscited by 22
  12. 2021Differentiable Network Pruning for Microcontrollerscited by 15
35 more
  1. 2022GhostNets on Heterogeneous Devices via Cheap Operationscited by 14
  2. 2022Attentive Fine-Grained Structured Sparsity for Image Restorationcited by 12
  3. 2023PDP: Parameter-free Differentiable Pruning is All You Needcited by 12
  4. 2021SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learningcited by 11
  5. 2024ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networkscited by 11
  6. 2023Differentiable Transportation Pruningcited by 10
  7. 2024An Effective Information Theoretic Framework for Channel Pruningcited by 7
  8. 2022Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruningcited by 5
  9. 2022Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANscited by 5
  10. 2025On Accelerating Edge AI: Optimizing Resource-Constrained Environmentscited by 5
  11. 2023Progressive Channel-Shrinking Networkcited by 4
  12. 2024REPrune: Channel Pruning via Kernel Representative Selectioncited by 4
  13. 2023Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolutioncited by 3
  14. 2021Batch Normalization Tells You Which Filter is Importantcited by 2
  15. 2023Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Samplercited by 2
  16. 2024Differentiable Model Scaling using Differentiable Topkcited by 2
  17. 2025Deep Lookup Networkcited by 2
  18. 2021BWCP: Probabilistic Learning-to-Prune Channels for ConvNets via Batch Whiteningcited by 1
  19. 2021Evolving Transferable Neural Pruning Functionscited by 1
  20. 2021Exploring Gradient Flow Based Saliency for DNN Model Compressioncited by 1
  21. 2022STN: Scalable Tensorizing Networks via Structure-Aware Training and Adaptive Compressioncited by 1
  22. 2023SUBP: Soft Uniform Block Pruning for 1xN Sparse CNNs Multithreading Accelerationcited by 1
  23. 2024Separate, Dynamic and Differentiable (SMART) Pruner for Block/Output Channel Pruning on Computer Vision Taskscited by 1
  24. 2025Advancing Weight and Channel Sparsification with Enhanced Saliencycited by 1
  25. 2021AIP: Adversarial Iterative Pruning Based on Knowledge Transfer for Convolutional Neural Networkscited by 0
  26. 2021Architecture Aware Latency Constrained Sparse Neural Networkscited by 0
  27. 2021ADDS: Adaptive Differentiable Sampling for Robust Multi-Party Learningcited by 0
  28. 2021Self-supervised Feature-Gate Coupling for Dynamic Network Pruningcited by 0
  29. 2023CR-SFP: Learning Consistent Representation for Soft Filter Pruningcited by 0
  30. 2024Convolutional Neural Network Compression via Dynamic Parameter Rank Pruningcited by 0
  31. 2024Towards Generalized Entropic Sparsification for Convolutional Neural Networkscited by 0
  32. 2025Playing the Lottery With Concave Regularizers for Sparse Trainable Neural Networkscited by 0
  33. 2025Exploring Vision Neural Network Pruning via Screening Methodologycited by 0
  34. 2025The Effects of Grouped Structural Global Pruning of Vision Transformers on Domain Generalisationcited by 0
  35. 2025Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learningcited by 0