# Network Decoupling: From Regular to Depthwise Separable Convolutions

arXiv: https://arxiv.org/abs/1808.05517 · HTML: https://mlsanity.com/paper/1808.05517/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2018.

## Built on by (most cited first, 18 of 18)

- [Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition](https://mlsanity.com/paper/2106.15125/) (2021)
- [Sparse Training via Boosting Pruning Plasticity with Neuroregeneration](https://mlsanity.com/paper/2106.10404/) (2021)
- [A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services](https://mlsanity.com/paper/2304.14271/) (2023)
- [Measuring the Ripeness of Fruit with Hyperspectral Imaging and Deep Learning](https://mlsanity.com/paper/2104.09808/) (2021)
- [Joint Matrix Decomposition for Deep Convolutional Neural Networks Compression](https://mlsanity.com/paper/2107.04386/) (2021)
- [Practical Network Acceleration with Tiny Sets](https://mlsanity.com/paper/2202.07861/) (2022)
- [PELA: Learning Parameter-Efficient Models with Low-Rank Approximation](https://mlsanity.com/paper/2310.10700/) (2023)
- [TinyAD: Memory-efficient anomaly detection for time series data in Industrial IoT](https://mlsanity.com/paper/2303.03611/) (2023)
- [Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks](https://mlsanity.com/paper/2505.20132/) (2025)
- [Real-Time Streamable Generative Speech Restoration with Flow Matching](https://mlsanity.com/paper/2512.19442/) (2025)
- [Semi-tensor Product-based TensorDecomposition for Neural Network Compression](https://mlsanity.com/paper/2109.15200/) (2021)
- [Practical Network Acceleration with Tiny Sets: Hypothesis, Theory, and Algorithm](https://mlsanity.com/paper/2303.00972/) (2023)
- [AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologies](https://mlsanity.com/paper/2402.13152/) (2024)
- [Transformer-Driven Neural Beamforming with Imperfect CSI in Urban Macro Wireless Channels](https://mlsanity.com/paper/2504.11667/) (2025)
- [Implicit Feature Decoupling with Depthwise Quantization](https://mlsanity.com/paper/2203.08080/) (2022)
- [On Unsupervised Image-to-image translation and GAN stability](https://mlsanity.com/paper/2403.09646/) (2023)
- [Continual Learning Beyond Experience Rehearsal and Full Model Surrogates](https://mlsanity.com/paper/2505.21942/) (2025)
- [LightVAE: Towards Compact and Efficient Video Autoencoders](https://mlsanity.com/paper/2610.14726/) (2026)

## Abstract (arXiv)

Depthwise separable convolution has shown great efficiency in network design, but requires time-consuming training procedure with full training-set available. This paper first analyzes the mathematical relationship between regular convolutions and depthwise separable convolutions, and proves that the former one could be approximated with the latter one in closed form. We show depthwise separable convolutions are principal components of regular convolutions. And then we propose network decoupling (ND), a training-free method to accelerate convolutional neural networks (CNNs) by transferring pre-trained CNN models into the MobileNet-like depthwise separable convolution structure, with a promising speedup yet negligible accuracy loss. We further verify through experiments that the proposed method is orthogonal to other training-free methods like channel decomposition, spatial decomposition, etc. Combining the proposed method with them will bring even larger CNN speedup. For instance, ND itself achieves about 2X speedup for the widely used VGG16, and combined with other methods, it reaches 3.7X speedup with graceful accuracy degradation. We demonstrate that ND is widely applicable to classification networks like ResNet, and object detection network like SSD300.