# Deep Neural Networks Based Weight Approximation and Computation Reuse for 2-D Image Classification

arXiv: https://arxiv.org/abs/2105.02954 · HTML: https://mlsanity.com/paper/2105.02954/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning](https://mlsanity.com/paper/1611.05128/) (2016)
- [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018)
- [Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?](https://mlsanity.com/paper/1907.02124/) (2019)
- [UCNN: Exploiting Computational Reuse in Deep Neural Networks via Weight Repetition](https://mlsanity.com/paper/1804.06508/) (2018)
- [2PFPCE: Two-Phase Filter Pruning Based on Conditional Entropy](https://mlsanity.com/paper/1809.02220/) (2018)
- [Guided Dropout](https://mlsanity.com/paper/1812.03965/) (2018)
- [The Effects of Approximate Multiplication on Convolutional Neural Networks](https://mlsanity.com/paper/2007.10500/) (2020)
- [Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations](https://mlsanity.com/paper/1703.03073/) (2017)
- [AutoTune: Automatically Tuning Convolutional Neural Networks for Improved Transfer Learning](https://mlsanity.com/paper/2005.02165/) (2020)
- [Accelerated CNN Training Through Gradient Approximation](https://mlsanity.com/paper/1908.05460/) (2019)

## Abstract (arXiv)

Deep Neural Networks (DNNs) are computationally and memory intensive, which makes their hardware implementation a challenging task especially for resource constrained devices such as IoT nodes. To address this challenge, this paper introduces a new method to improve DNNs performance by fusing approximate computing with data reuse techniques to be used for image recognition applications. DNNs weights are approximated based on the linear and quadratic approximation methods during the training phase, then, all of the weights are replaced with the linear/quadratic coefficients to execute the inference in a way where different weights could be computed using the same coefficients. This leads to a repetition of the weights across the processing element (PE) array, which in turn enables the reuse of the DNN sub-computations (computational reuse) and leverage the same data (data reuse) to reduce DNNs computations, memory accesses, and improve energy efficiency albeit at the cost of increased training time. Complete analysis for both MNIST and CIFAR 10 datasets is presented for image recognition , where LeNet 5 revealed a reduction in the number of parameters by a factor of 1211.3x with a drop of less than 0.9% in accuracy. When compared to the state of the art Row Stationary (RS) method, the proposed architecture saved 54% of the total number of adders and multipliers needed. Overall, the proposed approach is suitable for IoT edge devices as it reduces the memory size requirement as well as the number of needed memory accesses.