mlsanityData to 7 Oct 2026

Progressive Channel-Shrinking Network

arXiv 2304.00280 ↗First posted 1 Apr 2023cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Currently, salience-based channel pruning makes continuous breakthroughs in network compression. In the realization, the salience mechanism is used as a metric of channel salience to guide pruning. Therefore, salience-based channel pruning can dynamically adjust the channel width at run-time, which provides a flexible pruning scheme. However, there are two problems emerging: a gating function is often needed to truncate the specific salience entries to zero, which destabilizes the forward propagation; dynamic architecture brings more cost for indexing in inference which bottlenecks the inference speed. In this paper, we propose a Progressive Channel-Shrinking (PCS) method to compress the selected salience entries at run-time instead of roughly approximating them to zero. We also propose a Running Shrinking Policy to provide a testing-static pruning scheme that can reduce the memory access cost for filter indexing. We evaluate our method on ImageNet and CIFAR10 datasets over two prevalent networks: ResNet and VGG, and demonstrate that our PCS outperforms all baselines and achieves state-of-the-art in terms of compression-performance tradeoff. Moreover, we observe a significant and practical acceleration of inference.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 48 of its 67 references that are in the map. First posted on arXiv 1 Apr 2023.

Builds on

48 of 67 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2017Squeeze-and-Excitation Networkscited by 4,763
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  10. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  11. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  12. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
36 more
  1. 2019Searching for MobileNetV3cited by 1,984
  2. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  3. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  4. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  5. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  6. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  7. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  8. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  9. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  10. 2021Dynamic Neural Networks: A Surveycited by 405
  11. 2017Residual Attention Network for Image Classificationcited by 385
  12. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  13. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  14. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  15. 2018BAM: Bottleneck Attention Modulecited by 171
  16. 2017Convolutional Networks with Adaptive Inference Graphscited by 147
  17. 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
  18. 2017CondenseNet: An Efficient DenseNet using Learned Group Convolutionscited by 121
  19. 2020VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutionscited by 115
  20. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  21. 2019Provable Filter Pruning for Efficient Neural Networkscited by 98
  22. 2021Dynamic Slimmable Networkcited by 83
  23. 2018Channel Gating Neural Networkscited by 80
  24. 2020Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compressioncited by 72
  25. 2015Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answeringcited by 66
  26. 2017More is Less: A More Complicated Network with Less Inference Complexitycited by 66
  27. 2020DHP: Differentiable Meta Pruning via HyperNetworkscited by 57
  28. 2020DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocationcited by 47
  29. 2018You Look Twice: GaterNet for Dynamic Filter Selection in CNNscited by 46
  30. 2021Manifold Regularized Dynamic Network Pruningcited by 43
  31. 2019Batch-Shaping for Learning Conditional Channel Gated Networkscited by 42
  32. 2020Fully Dynamic Inference with Deep Neural Networkscited by 19
  33. 2020Dynamic Group Convolution for Accelerating Convolutional Neural Networkscited by 15
  34. 2018IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networkscited by 14
  35. 2018IGCV2: Interleaved Structured Sparse Convolutional Neural Networkscited by 12
  36. 2019Self-Adaptive Network Pruningcited by 7

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2024PSE-Net: Channel Pruning for Convolutional Neural Networks with Parallel-subnets Estimatorcited by 0
  2. 2025SOLAR: Switchable Output Layer for Accuracy and Robustness in Once-for-All Trainingcited by 0
  3. 2025Vanishing Contributions: A Unified Framework for Smooth and Iterative Model Compressioncited by 0
  4. 2025Optimally Deep Networks -- Adapting Model Depth to Datasets for Superior Efficiencycited by 0