# Pruning artificial neural networks: a way to find well-generalizing, high-entropy sharp minima

arXiv: https://arxiv.org/abs/2004.14765 · HTML: https://mlsanity.com/paper/2004.14765/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2020.

## Built on by (most cited first, 8 of 8)

- [SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks](https://mlsanity.com/paper/2102.03773/) (2021)
- [The rise of the lottery heroes: why zero-shot pruning is hard](https://mlsanity.com/paper/2202.12400/) (2022)
- [Bringing the Discussion of Minima Sharpness to the Audio Domain: a Filter-Normalised Evaluation for Acoustic Scene Classification](https://mlsanity.com/paper/2309.16369/) (2023)
- [Towards Efficient Capsule Networks](https://mlsanity.com/paper/2208.09203/) (2022)
- [SCoTTi: Save Computation at Training Time with an adaptive framework](https://mlsanity.com/paper/2312.12483/) (2023)
- [Hardware-Aware YOLO Compression for Low-Power Edge AI on STM32U5 for Weeds Detection in Digital Agriculture](https://mlsanity.com/paper/2511.07990/) (2025)
- [Routing the Lottery: Adaptive Subnetworks for Heterogeneous Data](https://mlsanity.com/paper/2601.22141/) (2026)
- [WaterMAS: Sharpness-Aware Maximization for Neural Network Watermarking](https://mlsanity.com/paper/2409.03902/) (2024)

## Abstract (arXiv)

Recently, a race towards the simplification of deep networks has begun, showing that it is effectively possible to reduce the size of these models with minimal or no performance loss. However, there is a general lack in understanding why these pruning strategies are effective. In this work, we are going to compare and analyze pruned solutions with two different pruning approaches, one-shot and gradual, showing the higher effectiveness of the latter. In particular, we find that gradual pruning allows access to narrow, well-generalizing minima, which are typically ignored when using one-shot approaches. In this work we also propose PSP-entropy, a measure to understand how a given neuron correlates to some specific learned classes. Interestingly, we observe that the features extracted by iteratively-pruned models are less correlated to specific classes, potentially making these models a better fit in transfer learning approaches.