# A Pre-defined Sparse Kernel Based Convolution for Deep CNNs

arXiv: https://arxiv.org/abs/1910.00724 · HTML: https://mlsanity.com/paper/1910.00724/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2019.

## Built on by (most cited first, 4 of 4)

- [Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edge](https://mlsanity.com/paper/2208.13092/) (2022)
- [HYPER-SNN: Towards Energy-efficient Quantized Deep Spiking Neural Networks for Hyperspectral Image Classification](https://mlsanity.com/paper/2107.11979/) (2021)
- [Artificial Neural Networks generated by Low Discrepancy Sequences](https://mlsanity.com/paper/2103.03543/) (2021)
- [Multimodal deep learning for mapping forest dominant height by fusing GEDI with earth observation data](https://mlsanity.com/paper/2311.11777/) (2023)

## Abstract (arXiv)

The high demand for computational and storage resources severely impede the deployment of deep convolutional neural networks (CNNs) in limited-resource devices. Recent CNN architectures have proposed reduced complexity versions (e.g. SuffleNet and MobileNet) but at the cost of modest decreases inaccuracy. This paper proposes pSConv, a pre-defined sparse 2D kernel-based convolution, which promises significant improvements in the trade-off between complexity and accuracy for both CNN training and inference. To explore the potential of this approach, we have experimented with two widely accepted datasets, CIFAR-10 and Tiny ImageNet, in sparse variants of both the ResNet18 and VGG16 architectures. Our approach shows a parameter count reduction of up to 4.24x with modest degradation in classification accuracy relative to that of standard CNNs. Our approach outperforms a popular variant of ShuffleNet using a variant of ResNet18 with pSConv having 3x3 kernels with only four of nine elements not fixed at zero. In particular, the parameter count is reduced by 1.7x for CIFAR-10 and 2.29x for Tiny ImageNet with an increased accuracy of ~4%.