# Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks

arXiv: https://arxiv.org/abs/2011.14356 · HTML: https://mlsanity.com/paper/2011.14356/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2020.

## Built on by (most cited first, 18 of 18)

- [Understanding the Dynamics of DNNs Using Graph Modularity](https://mlsanity.com/paper/2111.12485/) (2021)
- [SGLP: A Similarity Guided Fast Layer Partition Pruning for Compressing Large Deep Models](https://mlsanity.com/paper/2410.14720/) (2024)
- [DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks](https://mlsanity.com/paper/2206.00843/) (2022)
- [Layer Folding: Neural Network Depth Reduction using Activation Linearization](https://mlsanity.com/paper/2106.09309/) (2021)
- [SR-init: An interpretable layer pruning method](https://mlsanity.com/paper/2303.07677/) (2023)
- [UPDP: A Unified Progressive Depth Pruner for CNN and Vision Transformer](https://mlsanity.com/paper/2401.06426/) (2024)
- [Restructurable Activation Networks](https://mlsanity.com/paper/2208.08562/) (2022)
- [Concurrent Training and Layer Pruning of Deep Neural Networks](https://mlsanity.com/paper/2406.04549/) (2024)
- [Temporal Action Detection Model Compression by Progressive Block Drop](https://mlsanity.com/paper/2503.16916/) (2025)
- [MDP: Multidimensional Vision Model Pruning with Latency Constraint](https://mlsanity.com/paper/2504.02168/) (2025)
- [Multi-Dimensional Pruning: Joint Channel, Layer and Block Pruning with Latency Constraint](https://mlsanity.com/paper/2406.12079/) (2024)
- [Efficient Latency-Aware CNN Depth Compression via Two-Stage Dynamic Programming](https://mlsanity.com/paper/2301.12187/) (2023)
- [Complexity-Aware Training of Deep Neural Networks for Optimal Structure Discovery](https://mlsanity.com/paper/2411.09127/) (2024)
- [Structure-Aware Automatic Channel Pruning by Searching with Graph Embedding](https://mlsanity.com/paper/2506.11469/) (2025)
- [IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning](https://mlsanity.com/paper/2507.14171/) (2025)
- [Neural expressiveness for beyond importance model compression](https://mlsanity.com/paper/2512.06440/) (2025)
- [HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers](https://mlsanity.com/paper/2603.12222/) (2026)
- [Learn&Drop: Fast Learning of CNNs based on Layer Dropping](https://mlsanity.com/paper/2604.23403/) (2026)

## Abstract (arXiv)

In order to deploy deep convolutional neural networks (CNNs) on resource-limited devices, many model pruning methods for filters and weights have been developed, while only a few to layer pruning. However, compared with filter pruning and weight pruning, the compact model obtained by layer pruning has less inference time and run-time memory usage when the same FLOPs and number of parameters are pruned because of less data moving in memory. In this paper, we propose a simple layer pruning method using fusible residual convolutional block (ResConv), which is implemented by inserting shortcut connection with a trainable information control parameter into a single convolutional layer. Using ResConv structures in training can improve network accuracy and train deep plain networks, and adds no additional computation during inference process because ResConv is fused to be an ordinary convolutional layer after training. For layer pruning, we convert convolutional layers of network into ResConv with a layer scaling factor. In the training process, the L1 regularization is adopted to make the scaling factors sparse, so that unimportant layers are automatically identified and then removed, resulting in a model of layer reduction. Our pruning method achieves excellent performance of compression and acceleration over the state-of-the-arts on different datasets, and needs no retraining in the case of low pruning rate. For example, with ResNet-110, we achieve a 65.5%-FLOPs reduction by removing 55.5% of the parameters, with only a small loss of 0.13% in top-1 accuracy on CIFAR-10.