# Low-Energy Convolutional Neural Networks (CNNs) using Hadamard Method

arXiv: https://arxiv.org/abs/2209.09106 · HTML: https://mlsanity.com/paper/2209.09106/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 6 Sep 2022.

## Builds on (most cited first, 6 of 6)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Fast Training of Convolutional Networks through FFTs](https://mlsanity.com/paper/1312.5851/) (2013)
- [Efficient Sparse-Winograd Convolutional Neural Networks](https://mlsanity.com/paper/1802.06367/) (2018)
- [Energy Efficient Hadamard Neural Networks](https://mlsanity.com/paper/1805.05421/) (2018)
- [Performance Analysis of Semi-supervised Learning in the Small-data Regime using VAEs](https://mlsanity.com/paper/2002.12164/) (2020)

## Abstract (arXiv)

The growing demand for the internet of things (IoT) makes it necessary to implement computer vision tasks such as object recognition in low-power devices. Convolutional neural networks (CNNs) are a potential approach for object recognition and detection. However, the convolutional layer in CNN consumes significant energy compared to the fully connected layers. To mitigate this problem, a new approach based on the Hadamard transformation as an alternative to the convolution operation is demonstrated using two fundamental datasets, MNIST and CIFAR10. The mathematical expression of the Hadamard method shows the clear potential to save energy consumption compared to convolutional layers, which are helpful with BigData applications. In addition, to the test accuracy of the MNIST dataset, the Hadamard method performs similarly to the convolution method. In contrast, with the CIFAR10 dataset, test data accuracy is dropped (due to complex data and multiple channels) compared to the convolution method. Finally, the demonstrated method is helpful for other computer vision tasks when the kernel size is smaller than the input image size.