# SR-init: An interpretable layer pruning method

arXiv: https://arxiv.org/abs/2303.07677 · HTML: https://mlsanity.com/paper/2303.07677/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 26 references that are in the map.
- First posted on arXiv 14 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Importance Estimation for Neural Network Pruning](https://mlsanity.com/paper/1906.10771/) (2019)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Data-Driven Sparse Structure Selection for Deep Neural Networks](https://mlsanity.com/paper/1707.01213/) (2017)
- [Residual Connections Encourage Iterative Inference](https://mlsanity.com/paper/1710.04773/) (2017)
- [Understanding the Dynamics of DNNs Using Graph Modularity](https://mlsanity.com/paper/2111.12485/) (2021)
- [DBP: Discrimination Based Block-Level Pruning for Deep Model Acceleration](https://mlsanity.com/paper/1912.10178/) (2019)
- [To Filter Prune, or to Layer Prune, That Is The Question](https://mlsanity.com/paper/2007.05667/) (2020)
- [Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks](https://mlsanity.com/paper/2011.14356/) (2020)

## Built on by (most cited first, 13 of 13)

- [SepPrune: Structured Pruning for Efficient Deep Speech Separation](https://mlsanity.com/paper/2505.12079/) (2025)
- [SGLP: A Similarity Guided Fast Layer Partition Pruning for Compressing Large Deep Models](https://mlsanity.com/paper/2410.14720/) (2024)
- [Reassessing Layer Pruning in LLMs: New Insights and Methods](https://mlsanity.com/paper/2411.15558/) (2024)
- [A Generic Layer Pruning Method for Signal Modulation Recognition Deep Learning Models](https://mlsanity.com/paper/2406.07929/) (2024)
- [MDP: Multidimensional Vision Model Pruning with Latency Constraint](https://mlsanity.com/paper/2504.02168/) (2025)
- [Multi-Dimensional Pruning: Joint Channel, Layer and Block Pruning with Latency Constraint](https://mlsanity.com/paper/2406.12079/) (2024)
- [RedTest: Towards Measuring Redundancy in Deep Neural Networks Effectively](https://mlsanity.com/paper/2411.10507/) (2024)
- [FCOS: A Two-Stage Recoverable Model Pruning Framework for Automatic Modulation Recognition](https://mlsanity.com/paper/2505.21571/) (2025)
- [Block Pruning for Enhanced Efficiency in Convolutional Neural Networks](https://mlsanity.com/paper/2312.16904/) (2023)
- [Multi-Modal Machine Learning Framework for Predicting Early Recurrence of Brain Tumors Using MRI and Clinical Biomarkers](https://mlsanity.com/paper/2509.01161/) (2025)
- [A Multimodal Deep Learning Framework for Early Diagnosis of Liver Cancer via Optimized BiLSTM-AM-VMD Architecture](https://mlsanity.com/paper/2509.01164/) (2025)
- [HSCP: A Two-Stage Spectral Clustering Framework for Resource-Constrained UAV Identification](https://mlsanity.com/paper/2512.08983/) (2025)
- [HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers](https://mlsanity.com/paper/2603.12222/) (2026)

## Abstract (arXiv)

Despite the popularization of deep neural networks (DNNs) in many fields, it is still challenging to deploy state-of-the-art models to resource-constrained devices due to high computational overhead. Model pruning provides a feasible solution to the aforementioned challenges. However, the interpretation of existing pruning criteria is always overlooked. To counter this issue, we propose a novel layer pruning method by exploring the Stochastic Re-initialization. Our SR-init method is inspired by the discovery that the accuracy drop due to stochastic re-initialization of layer parameters differs in various layers. On the basis of this observation, we come up with a layer pruning criterion, i.e., those layers that are not sensitive to stochastic re-initialization (low accuracy drop) produce less contribution to the model and could be pruned with acceptable loss. Afterward, we experimentally verify the interpretability of SR-init via feature visualization. The visual explanation demonstrates that SR-init is theoretically feasible, thus we compare it with state-of-the-art methods to further evaluate its practicability. As for ResNet56 on CIFAR-10 and CIFAR-100, SR-init achieves a great reduction in parameters (63.98% and 37.71%) with an ignorable drop in top-1 accuracy (-0.56% and 0.8%). With ResNet50 on ImageNet, we achieve a 15.59% FLOPs reduction by removing 39.29% of the parameters, with only a drop of 0.6% in top-1 accuracy. Our code is available at https://github.com/huitang-zjut/SR-init.