# OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks

arXiv: https://arxiv.org/abs/1905.11664 · HTML: https://mlsanity.com/paper/1905.11664/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 18 of 18)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation](https://mlsanity.com/paper/2303.17225/) (2023)
- [Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition](https://mlsanity.com/paper/2107.11442/) (2021)
- [GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization](https://mlsanity.com/paper/2109.02220/) (2021)
- [Activation Modulation and Recalibration Scheme for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2112.08996/) (2021)
- [Online Multi-Granularity Distillation for GAN Compression](https://mlsanity.com/paper/2108.06908/) (2021)
- [On the Effect of Pruning on Adversarial Robustness](https://mlsanity.com/paper/2108.04890/) (2021)
- [Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition](https://mlsanity.com/paper/2109.14710/) (2021)
- [Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion](https://mlsanity.com/paper/2107.06916/) (2021)
- [Automatic Sparse Connectivity Learning for Neural Networks](https://mlsanity.com/paper/2201.05020/) (2022)
- [Model compression as constrained optimization, with application to neural nets. Part V: combining compressions](https://mlsanity.com/paper/2107.04380/) (2021)
- [One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression](https://mlsanity.com/paper/2508.13836/) (2025)
- [SMOF: Squeezing More Out of Filters Yields Hardware-Friendly CNN Pruning](https://mlsanity.com/paper/2110.10842/) (2021)
- [Learning k-Level Structured Sparse Neural Networks Using Group Envelope Regularization](https://mlsanity.com/paper/2212.12921/) (2022)
- [Learning Accurate Performance Predictors for Ultrafast Automated Model Compression](https://mlsanity.com/paper/2304.06393/) (2023)
- [Mining the Weights Knowledge for Optimizing Neural Network Structures](https://mlsanity.com/paper/2110.05954/) (2021)
- [Catalyst: a Novel Regularizer for Structured Pruning with Auxiliary Extension of Parameter Space](https://mlsanity.com/paper/2507.14170/) (2025)
- [Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers](https://mlsanity.com/paper/2609.21126/) (2026)

## Abstract (arXiv)

Channel pruning can significantly accelerate and compress deep neural networks. Many channel pruning works utilize structured sparsity regularization to zero out all the weights in some channels and automatically obtain structure-sparse network in training stage. However, these methods apply structured sparsity regularization on each layer separately where the correlations between consecutive layers are omitted. In this paper, we first combine one out-channel in current layer and the corresponding in-channel in next layer as a regularization group, namely out-in-channel. Our proposed Out-In-Channel Sparsity Regularization (OICSR) considers correlations between successive layers to further retain predictive power of the compact network. Training with OICSR thoroughly transfers discriminative features into a fraction of out-in-channels. Correspondingly, OICSR measures channel importance based on statistics computed from two consecutive layers, not individual layer. Finally, a global greedy pruning algorithm is designed to remove redundant out-in-channels in an iterative way. Our method is comprehensively evaluated with various CNN architectures including CifarNet, AlexNet, ResNet, DenseNet and PreActSeNet on CIFAR-10, CIFAR-100 and ImageNet-1K datasets. Notably, on ImageNet-1K, we reduce 37.2% FLOPs on ResNet-50 while outperforming the original model by 0.22% top-1 accuracy.