Bayesian Optimization with Clustering and Rollback for CNN Auto Pruning
Not on the map. Why?
Abstract (arXiv)
Pruning is an effective technique for convolutional neural networks (CNNs) model compression, but it is difficult to find the optimal pruning policy due to the large design space. To improve the usability of pruning, many auto pruning methods have been developed. Recently, Bayesian optimization (BO) has been considered to be a competitive algorithm for auto pruning due to its solid theoretical foundation and high sampling efficiency. However, BO suffers from the curse of dimensionality. The performance of BO deteriorates when pruning deep CNNs, since the dimension of the design spaces increase. We propose a novel clustering algorithm that reduces the dimension of the design space to speed up the searching process. Subsequently, a rollback algorithm is proposed to recover the high-dimensional design space so that higher pruning accuracy can be obtained. We validate our proposed method on ResNet, MobileNetV1, and MobileNetV2 models. Experiments show that the proposed method significantly improves the convergence rate of BO when pruning deep CNNs with no increase in running time. The source code is available at https://github.com/fanhanwei/BOCR.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 49 references that are in the map. First posted on arXiv 22 Sep 2021.
Builds on
27 of 49 references.
References · 27
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
- 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
- 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
- 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
- 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
- 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
- 2016Dynamic Network Surgery for Efficient DNNscited by 233
- 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
15 more
- 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
- 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
- 2013Bayesian Optimization in a Billion Dimensions via Random Embeddingscited by 178
- 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
- 2015High Dimensional Bayesian Optimisation and Bandits via Additive Modelscited by 137
- 2015Data-free parameter pruning for Deep Neural Networkscited by 112
- 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
- 2021Convolutional Neural Network Pruning with Structural Redundancy Reductioncited by 73
- 2018High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groupscited by 58
- 2017Batched High-dimensional Bayesian Optimization via Structural Kernel Learningcited by 51
- 2019AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Ratescited by 49
- 2020Robust Pruning at Initializationcited by 31
- 2020Auto Graph Encoder-Decoder for Neural Network Pruningcited by 26
- 2017Fine-Pruning: Joint Fine-Tuning and Compression of a Convolutional Network with Bayesian Optimizationcited by 8
- 2020Additive Tree-Structured Conditional Parameter Spaces in Bayesian Optimization: A Novel Covariance Function and a Fast Implementationcited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.