# Comprehensive Online Network Pruning via Learnable Scaling Factors

arXiv: https://arxiv.org/abs/2010.02623 · HTML: https://mlsanity.com/paper/2010.02623/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 7 of 7)

- [Deep Model Compression based on the Training History](https://mlsanity.com/paper/2102.00160/) (2021)
- [Looking into Black Box Code Language Models](https://mlsanity.com/paper/2407.04868/) (2024)
- [Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attribution](https://mlsanity.com/paper/2502.06809/) (2025)
- [Self-Supervised Monocular Depth Estimation of Untextured Indoor Rotated Scenes](https://mlsanity.com/paper/2106.12958/) (2021)
- [Neural Network Pruning Through Constrained Reinforcement Learning](https://mlsanity.com/paper/2110.08558/) (2021)
- [Information Consistent Pruning: How to Efficiently Search for Sparse Networks?](https://mlsanity.com/paper/2501.15592/) (2025)
- [Multi-Granular Node Pruning for Causal Circuit Discovery](https://mlsanity.com/paper/2512.10903/) (2025)

## Abstract (arXiv)

One of the major challenges in deploying deep neural network architectures is their size which has an adverse effect on their inference time and memory requirements. Deep CNNs can either be pruned width-wise by removing filters based on their importance or depth-wise by removing layers and blocks. Width wise pruning (filter pruning) is commonly performed via learnable gates or switches and sparsity regularizers whereas pruning of layers has so far been performed arbitrarily by manually designing a smaller network usually referred to as a student network. We propose a comprehensive pruning strategy that can perform both width-wise as well as depth-wise pruning. This is achieved by introducing gates at different granularities (neuron, filter, layer, block) which are then controlled via an objective function that simultaneously performs pruning at different granularity during each forward pass. Our approach is applicable to wide-variety of architectures without any constraints on spatial dimensions or connection type (sequential, residual, parallel or inception). Our method has resulted in a compression ratio of 70% to 90% without noticeable loss in accuracy when evaluated on benchmark datasets.