mlsanityData to 7 Oct 2026

Soft Masking for Cost-Constrained Channel Pruning

arXiv 2211.02206 ↗First posted 4 Nov 2022cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Structured channel pruning has been shown to significantly accelerate inference time for convolution neural networks (CNNs) on modern hardware, with a relatively minor loss of network accuracy. Recent works permanently zero these channels during training, which we observe to significantly hamper final accuracy, particularly as the fraction of the network being pruned increases. We propose Soft Masking for cost-constrained Channel Pruning (SMCP) to allow pruned channels to adaptively return to the network while simultaneously pruning towards a target cost constraint. By adding a soft mask re-parameterization of the weights and channel pruning from the perspective of removing input channels, we allow gradient updates to previously pruned channels and the opportunity for the channels to later return to the network. We then formulate input channel pruning as a global resource allocation problem. Our method outperforms prior works on both the ImageNet classification and PASCAL VOC detection datasets.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 47 of its 58 references that are in the map. First posted on arXiv 4 Nov 2022.

Builds on

47 of 58 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  6. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  8. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  9. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  10. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  11. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  12. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
35 more
  1. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  2. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  3. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  4. 2018Rethinking the Value of Network Pruningcited by 595
  5. 2019Importance Estimation for Neural Network Pruningcited by 531
  6. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  7. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  8. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  9. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  10. 2014cuDNN: Efficient Primitives for Deep Learningcited by 240
  11. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  12. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  13. 2021Accelerating Sparse Deep Neural Networkscited by 213
  14. 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 210
  15. 2016Speed/accuracy trade-offs for modern convolutional object detectorscited by 198
  16. 2019Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterizationcited by 187
  17. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 177
  18. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 150
  19. 2018NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applicationscited by 146
  20. 2020Dynamic Model Pruning with Feedbackcited by 125
  21. 2019Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networkscited by 118
  22. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  23. 2020Neural Pruning via Growing Regularizationcited by 97
  24. 2020EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruningcited by 80
  25. 2018ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptationcited by 76
  26. 2020Operation-Aware Soft Channel Pruning using Differentiable Maskscited by 74
  27. 2016Learning the Number of Neurons in Deep Networkscited by 69
  28. 2020Progressive Skeletonization: Trimming more fat from a network at initializationcited by 64
  29. 2019Discovering Neural Wiringscited by 56
  30. 2018DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architecturescited by 40
  31. 2021BCNet: Searching for Network Width with Bilaterally Coupled Networkcited by 23
  32. 2021NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimizationcited by 13
  33. 2019AutoSlim: Towards One-Shot Architecture Search for Channel Numberscited by 10
  34. 2021HALP: Hardware-Aware Latency Pruningcited by 8
  35. 2021Search Spaces for Neural Model Trainingcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 3; 2025: 3.

12 papers.

  1. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  2. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  3. 2023Adaptive Sharpness-Aware Pruning for Robust Sparse Networkscited by 17
  4. 2023Differentiable Transportation Pruningcited by 10
  5. 2025Pruning for Sparse Diffusion Models based on Gradient Flowcited by 5
  6. 2024Multi-Dimensional Pruning: Joint Channel, Layer and Block Pruning with Latency Constraintcited by 4
  7. 2025MDP: Multidimensional Vision Model Pruning with Latency Constraintcited by 4
  8. 2024Differentiable Model Scaling using Differentiable Topkcited by 2
  9. 2024AutoSculpt: A Pattern-based Model Auto-pruning Framework Using Reinforcement Learning and Graph Learningcited by 1
  10. 2023MaxQ: Multi-Axis Query for N:M Sparsity Networkcited by 0
  11. 2023CR-SFP: Learning Consistent Representation for Soft Filter Pruningcited by 0
  12. 2025CoSeP: Complementary Separability Pruning via Class-Separability Clusteringcited by 0