# Network Pruning via Resource Reallocation

arXiv: https://arxiv.org/abs/2103.01847 · HTML: https://mlsanity.com/paper/2103.01847/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression](https://mlsanity.com/paper/1707.06342/) (2017)
- [AMC: AutoML for Model Compression and Acceleration on Mobile Devices](https://mlsanity.com/paper/1802.03494/) (2018)
- [Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration](https://mlsanity.com/paper/1811.00250/) (2018)
- [Structured Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/1903.04197/) (2019)
- [MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning](https://mlsanity.com/paper/1903.10258/) (2019)
- [Universally Slimmable Networks and Improved Training Techniques](https://mlsanity.com/paper/1903.05134/) (2019)
- [Discrimination-aware Channel Pruning for Deep Neural Networks](https://mlsanity.com/paper/1810.11809/) (2018)
- [Learning Lightweight Lane Detection CNNs by Self Attention Distillation](https://mlsanity.com/paper/1908.00821/) (2019)
- [SCOP: Scientific Control for Reliable Neural Network Pruning](https://mlsanity.com/paper/2010.10732/) (2020)
- [AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference](https://mlsanity.com/paper/1805.08941/) (2018)
- [Towards Efficient Model Compression via Learned Global Ranking](https://mlsanity.com/paper/1904.12368/) (2019)
- [Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection](https://mlsanity.com/paper/2003.01794/) (2020)
- [DMCP: Differentiable Markov Channel Pruning for Neural Networks](https://mlsanity.com/paper/2005.03354/) (2020)
- [Inter-Region Affinity Distillation for Road Marking Segmentation](https://mlsanity.com/paper/2004.05304/) (2020)
- [Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks](https://mlsanity.com/paper/1811.02759/) (2018)
- [Deeply-supervised Knowledge Synergy](https://mlsanity.com/paper/1906.00675/) (2019)
- [Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization](https://mlsanity.com/paper/1812.00481/) (2018)

## Built on by (most cited first, 6 of 6)

- [CHEX: CHannel EXploration for CNN Model Compression](https://mlsanity.com/paper/2203.15794/) (2022)
- [RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging](https://mlsanity.com/paper/2110.01397/) (2021)
- [DynamicKD: An Effective Knowledge Distillation via Dynamic Entropy Correction-Based Distillation for Gap Optimizing](https://mlsanity.com/paper/2305.05233/) (2023)
- [Self-supervised Feature-Gate Coupling for Dynamic Network Pruning](https://mlsanity.com/paper/2111.14302/) (2021)
- [Semantic-embedded Similarity Prototype for Scene Recognition](https://mlsanity.com/paper/2308.05896/) (2023)
- [PSE-Net: Channel Pruning for Convolutional Neural Networks with Parallel-subnets Estimator](https://mlsanity.com/paper/2408.16233/) (2024)

## Abstract (arXiv)

Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network. Contemporary methods typically perform iterative pruning procedure from the original over-parameterized model, which is both tedious and expensive especially when the pruning is aggressive. In this paper, we propose a simple yet effective channel pruning technique, termed network Pruning via rEsource rEalLocation (PEEL), to quickly produce a desired slim model with negligible cost. Specifically, PEEL first constructs a predefined backbone and then conducts resource reallocation on it to shift parameters from less informative layers to more important layers in one round, thus amplifying the positive effect of these informative layers. To demonstrate the effectiveness of PEEL , we perform extensive experiments on ImageNet with ResNet-18, ResNet-50, MobileNetV2, MobileNetV3-small and EfficientNet-B0. Experimental results show that structures uncovered by PEEL exhibit competitive performance with state-of-the-art pruning algorithms under various pruning settings. Our code is available at https://github.com/cardwing/Codes-for-PEEL.