# ResNet Can Be Pruned 60x: Introducing Network Purification and Unused Path Removal (P-RM) after Weight Pruning

arXiv: https://arxiv.org/abs/1905.00136 · HTML: https://mlsanity.com/paper/1905.00136/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2019.

## Built on by (most cited first, 13 of 13)

- [DepGraph: Towards Any Structural Pruning](https://mlsanity.com/paper/2301.12900/) (2023)
- [Teachers Do More Than Teach: Compressing Image-to-Image Models](https://mlsanity.com/paper/2103.03467/) (2021)
- [Isomorphic Pruning for Vision Models](https://mlsanity.com/paper/2407.04616/) (2024)
- [FORMS: Fine-grained Polarized ReRAM-based In-situ Computation for Mixed-signal DNN Accelerator](https://mlsanity.com/paper/2106.09144/) (2021)
- [Improving DNN Fault Tolerance using Weight Pruning and Differential Crossbar Mapping for ReRAM-based Edge AI](https://mlsanity.com/paper/2106.09166/) (2021)
- [Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration](https://mlsanity.com/paper/2111.11581/) (2021)
- [Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices](https://mlsanity.com/paper/2103.03483/) (2021)
- [Pruning vs XNOR-Net: A Comprehensive Study of Deep Learning for Audio Classification on Edge-devices](https://mlsanity.com/paper/2108.06128/) (2021)
- [FedCode: Communication-Efficient Federated Learning via Transferring Codebooks](https://mlsanity.com/paper/2311.09270/) (2023)
- [InDistill: Information flow-preserving knowledge distillation for model compression](https://mlsanity.com/paper/2205.10003/) (2022)
- [DARTS-PRIME: Regularization and Scheduling Improve Constrained Optimization in Differentiable NAS](https://mlsanity.com/paper/2106.11655/) (2021)
- [Subtractor-Based CNN Inference Accelerator](https://mlsanity.com/paper/2310.01022/) (2023)
- [Towards Generalized Entropic Sparsification for Convolutional Neural Networks](https://mlsanity.com/paper/2404.04734/) (2024)

## Abstract (arXiv)

The state-of-art DNN structures involve high computation and great demand for memory storage which pose intensive challenge on DNN framework resources. To mitigate the challenges, weight pruning techniques has been studied. However, high accuracy solution for extreme structured pruning that combines different types of structured sparsity still waiting for unraveling due to the extremely reduced weights in DNN networks. In this paper, we propose a DNN framework which combines two different types of structured weight pruning (filter and column prune) by incorporating alternating direction method of multipliers (ADMM) algorithm for better prune performance. We are the first to find non-optimality of ADMM process and unused weights in a structured pruned model, and further design an optimization framework which contains the first proposed Network Purification and Unused Path Removal algorithms which are dedicated to post-processing an structured pruned model after ADMM steps. Some high lights shows we achieve 232x compression on LeNet-5, 60x compression on ResNet-18 CIFAR-10 and over 5x compression on AlexNet. We share our models at anonymous link http://bit.ly/2VJ5ktv.