# The Effects of Approximate Multiplication on Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2007.10500 · HTML: https://mlsanity.com/paper/2007.10500/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2020.

## Built on by (most cited first, 10 of 10)

- [ApproxTrain: Fast Simulation of Approximate Multipliers for DNN Training and Inference](https://mlsanity.com/paper/2209.04161/) (2022)
- [Multiplication-Free Transformer Training via Piecewise Affine Operations](https://mlsanity.com/paper/2305.17190/) (2023)
- [An Alternative Practice of Tropical Convolution to Traditional Convolutional Neural Networks](https://mlsanity.com/paper/2103.02096/) (2021)
- [PLAM: a Posit Logarithm-Approximate Multiplier](https://mlsanity.com/paper/2102.09262/) (2021)
- [TabConv: Low-Computation CNN Inference via Table Lookups](https://mlsanity.com/paper/2404.05872/) (2024)
- [Deep Neural Networks Based Weight Approximation and Computation Reuse for 2-D Image Classification](https://mlsanity.com/paper/2105.02954/) (2021)
- [A Survey on Design Methodologies for Accelerating Deep Learning on Heterogeneous Architectures](https://mlsanity.com/paper/2311.17815/) (2023)
- [Unified Kernel-Segregated Transpose Convolution Operation](https://mlsanity.com/paper/2502.20493/) (2025)
- [Precision-Aware Variable Bit Processing Elements for Hardware-Efficient Systolic Array Designs](https://mlsanity.com/paper/2608.22378/) (2026)
- [FAME: An FPGA-Based Platform for Approximate Multipliers Evaluation with Pattern-Guided DNN Retraining](https://mlsanity.com/paper/2609.17730/) (2026)

## Abstract (arXiv)

This paper analyzes the effects of approximate multiplication when performing inferences on deep convolutional neural networks (CNNs). The approximate multiplication can reduce the cost of the underlying circuits so that CNN inferences can be performed more efficiently in hardware accelerators. The study identifies the critical factors in the convolution, fully-connected, and batch normalization layers that allow more accurate CNN predictions despite the errors from approximate multiplication. The same factors also provide an arithmetic explanation of why bfloat16 multiplication performs well on CNNs. The experiments are performed with recognized network architectures to show that the approximate multipliers can produce predictions that are nearly as accurate as the FP32 references, without additional training. For example, the ResNet and Inception-v4 models with Mitch-w6 multiplication produces Top-5 errors that are within 0.2% compared to the FP32 references. A brief cost comparison of Mitch-w6 against bfloat16 is presented, where a MAC operation saves up to 80% of energy compared to the bfloat16 arithmetic. The most far-reaching contribution of this paper is the analytical justification that multiplications can be approximated while additions need to be exact in CNN MAC operations.