# Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions

arXiv: https://arxiv.org/abs/1803.09926 · HTML: https://mlsanity.com/paper/1803.09926/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2018.

## Built on by (most cited first, 6 of 6)

- [Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AI](https://mlsanity.com/paper/2309.02065/) (2023)
- [Learn to Code Sustainably: An Empirical Study on LLM-based Green Code Generation](https://mlsanity.com/paper/2403.03344/) (2024)
- [RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network](https://mlsanity.com/paper/2206.14098/) (2022)
- [Caterpillar: A Pure-MLP Architecture with Shifted-Pillars-Concatenation](https://mlsanity.com/paper/2305.17644/) (2023)
- [Swan: A Neural Engine for Efficient DNN Training on Smartphone SoCs](https://mlsanity.com/paper/2206.04687/) (2022)
- [Accelerating the Training of Video Super-Resolution Models](https://mlsanity.com/paper/2205.05069/) (2022)

## Abstract (arXiv)

Depthwise convolutions provide significant performance benefits owing to the reduction in both parameters and mult-adds. However, training depthwise convolution layers with GPUs is slow in current deep learning frameworks because their implementations cannot fully utilize the GPU capacity. To address this problem, in this paper we present an efficient method (called diagonalwise refactorization) for accelerating the training of depthwise convolution layers. Our key idea is to rearrange the weight vectors of a depthwise convolution into a large diagonal weight matrix so as to convert the depthwise convolution into one single standard convolution, which is well supported by the cuDNN library that is highly-optimized for GPU computations. We have implemented our training method in five popular deep learning frameworks. Evaluation results show that our proposed method gains 15.4× training speedup on Darknet, 8.4× on Caffe, 5.4× on PyTorch, 3.5× on MXNet, and 1.4× on TensorFlow, compared to their original implementations of depthwise convolutions.