# Low-complexity Approximate Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2208.00087 · HTML: https://mlsanity.com/paper/2208.00087/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 101 references that are in the map.
- First posted on arXiv 29 Jul 2022.

## Builds on (most cited first, 19 of 19)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation](https://mlsanity.com/paper/1404.0736/) (2014)
- [Speeding up Convolutional Neural Networks with Low Rank Expansions](https://mlsanity.com/paper/1405.3866/) (2014)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks](https://mlsanity.com/paper/1312.6229/) (2013)
- [Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition](https://mlsanity.com/paper/1412.6553/) (2014)
- [Convolutional Neural Networks using Logarithmic Data Representation](https://mlsanity.com/paper/1603.01025/) (2016)
- [Fast Training of Convolutional Networks through FFTs](https://mlsanity.com/paper/1312.5851/) (2013)
- [Bitwise Neural Networks](https://mlsanity.com/paper/1601.06071/) (2016)
- [Deep Fried Convnets](https://mlsanity.com/paper/1412.7149/) (2014)
- [DCT-like Transform for Image Compression Requires 14 Additions Only](https://mlsanity.com/paper/1702.00817/) (2017)
- [Low-complexity 8-point DCT Approximations Based on Integer Functions](https://mlsanity.com/paper/1309.7604/) (2013)
- [Improved 8-point Approximate DCT for Image and Video Compression Requiring Only 14 Additions](https://mlsanity.com/paper/1501.02995/) (2015)
- [An Integer Approximation Method for Discrete Sinusoidal Transforms](https://mlsanity.com/paper/2007.02232/) (2020)
- [A Class of DCT Approximations Based on the Feig-Winograd Algorithm](https://mlsanity.com/paper/1502.00592/) (2015)
- [A Discrete Tchebichef Transform Approximation for Image and Video Coding](https://mlsanity.com/paper/1502.00555/) (2015)
- [Computational Cost Reduction in Learned Transform Classifications](https://mlsanity.com/paper/1504.06779/) (2015)

## Built on by (most cited first, 1 of 1)

- [SBPF: Sensitiveness Based Pruning Framework For Convolutional Neural Network On Image Classification](https://mlsanity.com/paper/2208.04588/) (2022)

## Abstract (arXiv)

In this paper, we present an approach for minimizing the computational complexity of trained Convolutional Neural Networks (ConvNet). The idea is to approximate all elements of a given ConvNet and replace the original convolutional filters and parameters (pooling and bias coefficients; and activation function) with efficient approximations capable of extreme reductions in computational complexity. Low-complexity convolution filters are obtained through a binary (zero-one) linear programming scheme based on the Frobenius norm over sets of dyadic rationals. The resulting matrices allow for multiplication-free computations requiring only addition and bit-shifting operations. Such low-complexity structures pave the way for low-power, efficient hardware designs. We applied our approach on three use cases of different complexity: (i) a "light" but efficient ConvNet for face detection (with around 1000 parameters); (ii) another one for hand-written digit classification (with more than 180000 parameters); and (iii) a significantly larger ConvNet: AlexNet with ≈1.2 million matrices. We evaluated the overall performance on the respective tasks for different levels of approximations. In all considered applications, very low-complexity approximations have been derived maintaining an almost equal classification performance.