# Methods for Pruning Deep Neural Networks

arXiv: https://arxiv.org/abs/2011.00241 · HTML: https://mlsanity.com/paper/2011.00241/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2020.

## Built on by (most cited first, 30 of 49)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Kolmogorov-Arnold Network for Satellite Image Classification in Remote Sensing](https://mlsanity.com/paper/2406.00600/) (2024)
- [REEF: Representation Encoding Fingerprints for Large Language Models](https://mlsanity.com/paper/2410.14273/) (2024)
- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN](https://mlsanity.com/paper/2403.03409/) (2024)
- [Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN](https://mlsanity.com/paper/2107.14444/) (2021)
- [Improving the Transferability of Adversarial Examples via Direction Tuning](https://mlsanity.com/paper/2303.15109/) (2023)
- [Reducing Computational Complexity of Neural Networks in Optical Channel Equalization: From Concepts to Implementation](https://mlsanity.com/paper/2208.12866/) (2022)
- [FPGA Resource-aware Structured Pruning for Real-Time Neural Networks](https://mlsanity.com/paper/2308.05170/) (2023)
- [Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion](https://mlsanity.com/paper/2107.06916/) (2021)
- [On the Compression of Neural Networks Using ℓ0-Norm Regularization and Weight Pruning](https://mlsanity.com/paper/2109.05075/) (2021)
- [Neural Network Pruning by Gradient Descent](https://mlsanity.com/paper/2311.12526/) (2023)
- [WeedNet: A Foundation Model-Based Global-to-Local AI Approach for Real-Time Weed Species Identification and Classification](https://mlsanity.com/paper/2505.18930/) (2025)
- [Enhancing Edge Intelligence with Highly Discriminant LNT Features](https://mlsanity.com/paper/2312.14968/) (2023)
- [Bayesian computation with generative diffusion models by Multilevel Monte Carlo](https://mlsanity.com/paper/2409.15511/) (2024)
- [On The Fairness Impacts of Hardware Selection in Machine Learning](https://mlsanity.com/paper/2312.03886/) (2023)
- [Only Send What You Need: Learning to Communicate Efficiently in Federated Multilingual Machine Translation](https://mlsanity.com/paper/2401.07456/) (2024)
- [Designing Extremely Memory-Efficient CNNs for On-device Vision Tasks](https://mlsanity.com/paper/2408.03663/) (2024)
- [MoENAS: Mixture-of-Expert based Neural Architecture Search for jointly Accurate, Fair, and Robust Edge Deep Neural Networks](https://mlsanity.com/paper/2502.07422/) (2025)
- [KairosAD: A SAM-Based Model for Industrial Anomaly Detection on Embedded Devices](https://mlsanity.com/paper/2505.24334/) (2025)
- [Adversarial Attacks on Machine Learning in Embedded and IoT Platforms](https://mlsanity.com/paper/2303.02214/) (2023)
- [Systematic construction of continuous-time neural networks for linear dynamical systems](https://mlsanity.com/paper/2403.16215/) (2024)
- [Confident magnitude-based neural network pruning](https://mlsanity.com/paper/2408.04759/) (2024)
- [Compressing Multi-Task Model for Autonomous Driving via Pruning and Knowledge Distillation](https://mlsanity.com/paper/2511.05557/) (2025)
- [Effective Fine-Tuning with Eigenvector Centrality Based Pruning](https://mlsanity.com/paper/2512.12543/) (2025)
- [Self-attention vector output similarities reveal how machines pay attention](https://mlsanity.com/paper/2512.21956/) (2025)
- [Theory of Minimal Weight Perturbations in Deep Networks and its Applications for Low-Rank Activated Backdoor Attacks](https://mlsanity.com/paper/2601.16880/) (2026)
- [Elimination-compensation pruning for fully-connected neural networks](https://mlsanity.com/paper/2602.20467/) (2026)

## Abstract (arXiv)

This paper presents a survey of methods for pruning deep neural networks. It begins by categorising over 150 studies based on the underlying approach used and then focuses on three categories: methods that use magnitude based pruning, methods that utilise clustering to identify redundancy, and methods that use sensitivity analysis to assess the effect of pruning. Some of the key influencing studies within these categories are presented to highlight the underlying approaches and results achieved. Most studies present results which are distributed in the literature as new architectures, algorithms and data sets have developed with time, making comparison across different studied difficult. The paper therefore provides a resource for the community that can be used to quickly compare the results from many different methods on a variety of data sets, and a range of architectures, including AlexNet, ResNet, DenseNet and VGG. The resource is illustrated by comparing the results published for pruning AlexNet and ResNet50 on ImageNet and ResNet56 and VGG16 on the CIFAR10 data to reveal which pruning methods work well in terms of retaining accuracy whilst achieving good compression rates. The paper concludes by identifying some promising directions for future research.