# Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1808.07471 · HTML: https://mlsanity.com/paper/1808.07471/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2018.

## Built on by (most cited first, 27 of 27)

- [Fast and Memory-Efficient Network Towards Efficient Image Super-Resolution](https://mlsanity.com/paper/2204.08397/) (2022)
- [Winning the Lottery Ahead of Time: Efficient Early Network Pruning](https://mlsanity.com/paper/2206.10451/) (2022)
- [A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services](https://mlsanity.com/paper/2304.14271/) (2023)
- [Green Federated Learning: A new era of Green Aware AI](https://mlsanity.com/paper/2409.12626/) (2024)
- [Deep Model Compression based on the Training History](https://mlsanity.com/paper/2102.00160/) (2021)
- [Fast and Accurate Road Crack Detection Based on Adaptive Cost-Sensitive Loss Function](https://mlsanity.com/paper/2106.15510/) (2021)
- [HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labels](https://mlsanity.com/paper/2202.10716/) (2022)
- [DynamicKD: An Effective Knowledge Distillation via Dynamic Entropy Correction-Based Distillation for Gap Optimizing](https://mlsanity.com/paper/2305.05233/) (2023)
- [WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks](https://mlsanity.com/paper/2302.08185/) (2023)
- [ABCP: Automatic Block-wise and Channel-wise Network Pruning via Joint Search](https://mlsanity.com/paper/2110.03858/) (2021)
- [Regularization-based Pruning of Irrelevant Weights in Deep Neural Architectures](https://mlsanity.com/paper/2204.04977/) (2022)
- [A Closer Look at Branch Classifiers of Multi-exit Architectures](https://mlsanity.com/paper/2204.13347/) (2022)
- [Recognition of Defective Mineral Wool Using Pruned ResNet Models](https://mlsanity.com/paper/2211.00466/) (2022)
- [Optimal Brain Connection: Towards Efficient Structural Pruning](https://mlsanity.com/paper/2508.05521/) (2025)
- [Batch Normalization Tells You Which Filter is Important](https://mlsanity.com/paper/2112.01155/) (2021)
- [Attaining Class-level Forgetting in Pretrained Model using Few Samples](https://mlsanity.com/paper/2210.10670/) (2022)
- [A Differentiable Framework for End-to-End Learning of Hybrid Structured Compression](https://mlsanity.com/paper/2309.13077/) (2023)
- [Decay Pruning Method: Smooth Pruning With a Self-Rectifying Procedure](https://mlsanity.com/paper/2406.03879/) (2024)
- [Weight Evolution: Improving Deep Neural Networks Training through Evolving Inferior Weight Values](https://mlsanity.com/paper/2110.04492/) (2021)
- [ADDS: Adaptive Differentiable Sampling for Robust Multi-Party Learning](https://mlsanity.com/paper/2110.15522/) (2021)
- [Asymptotic Soft Cluster Pruning for Deep Neural Networks](https://mlsanity.com/paper/2206.08186/) (2022)
- [Sub-network Multi-objective Evolutionary Algorithm for Filter Pruning](https://mlsanity.com/paper/2211.01957/) (2022)
- [Protective Self-Adaptive Pruning to Better Compress DNNs](https://mlsanity.com/paper/2303.11881/) (2023)
- [A Multi-objective Complex Network Pruning Framework Based on Divide-and-conquer and Global Performance Impairment Ranking](https://mlsanity.com/paper/2303.16212/) (2023)
- [Reduced storage direct tensor ring decomposition for convolutional neural networks compression](https://mlsanity.com/paper/2405.10802/) (2024)
- [SirenPose: Dynamic Scene Reconstruction via Geometric Supervision](https://mlsanity.com/paper/2512.20531/) (2025)
- [Learning Dynamic Scene Reconstruction with Sinusoidal Geometric Priors](https://mlsanity.com/paper/2512.22295/) (2025)

## Abstract (arXiv)

Deeper and wider Convolutional Neural Networks (CNNs) achieve superior performance but bring expensive computation cost. Accelerating such over-parameterized neural network has received increased attention. A typical pruning algorithm is a three-stage pipeline, i.e., training, pruning, and retraining. Prevailing approaches fix the pruned filters to zero during retraining, and thus significantly reduce the optimization space. Besides, they directly prune a large number of filters at first, which would cause unrecoverable information loss. To solve these problems, we propose an Asymptotic Soft Filter Pruning (ASFP) method to accelerate the inference procedure of the deep neural networks. First, we update the pruned filters during the retraining stage. As a result, the optimization space of the pruned model would not be reduced but be the same as that of the original model. In this way, the model has enough capacity to learn from the training data. Second, we prune the network asymptotically. We prune few filters at first and asymptotically prune more filters during the training procedure. With asymptotic pruning, the information of the training set would be gradually concentrated in the remaining filters, so the subsequent training and pruning process would be stable. Experiments show the effectiveness of our ASFP on image classification benchmarks. Notably, on ILSVRC-2012, our ASFP reduces more than 40% FLOPs on ResNet-50 with only 0.14% top-5 accuracy degradation, which is higher than the soft filter pruning (SFP) by 8%.