# Depth-wise Decomposition for Accelerating Separable Convolutions in Efficient Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1910.09455 · HTML: https://mlsanity.com/paper/1910.09455/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2019.

## Built on by (most cited first, 6 of 6)

- [Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling](https://mlsanity.com/paper/2406.07522/) (2024)
- [Novel View Synthesis from a Single RGBD Image for Indoor Scenes](https://mlsanity.com/paper/2311.01065/) (2023)
- [Faster Convolution Inference Through Using Pre-Calculated Lookup Tables](https://mlsanity.com/paper/2104.01681/) (2021)
- [Pruning Very Deep Neural Network Channels for Efficient Inference](https://mlsanity.com/paper/2211.08339/) (2022)
- [EucliDreamer: Fast and High-Quality Texturing for 3D Models with Depth-Conditioned Stable Diffusion](https://mlsanity.com/paper/2404.10279/) (2024)
- [VLCQ: Post-training quantization for deep neural networks using variable length coding](https://mlsanity.com/paper/2610.15092/) (2026)

## Abstract (arXiv)

Very deep convolutional neural networks (CNNs) have been firmly established as the primary methods for many computer vision tasks. However, most state-of-the-art CNNs are large, which results in high inference latency. Recently, depth-wise separable convolution has been proposed for image recognition tasks on computationally limited platforms such as robotics and self-driving cars. Though it is much faster than its counterpart, regular convolution, accuracy is sacrificed. In this paper, we propose a novel decomposition approach based on SVD, namely depth-wise decomposition, for expanding regular convolutions into depthwise separable convolutions while maintaining high accuracy. We show our approach can be further generalized to the multi-channel and multi-layer cases, based on Generalized Singular Value Decomposition (GSVD) [59]. We conduct thorough experiments with the latest ShuffleNet V2 model [47] on both random synthesized dataset and a large-scale image recognition dataset: ImageNet [10]. Our approach outperforms channel decomposition [73] on all datasets. More importantly, our approach improves the Top-1 accuracy of ShuffleNet V2 by ~2%.