# ApproxTrain: Fast Simulation of Approximate Multipliers for DNN Training and Inference

arXiv: https://arxiv.org/abs/2209.04161 · HTML: https://mlsanity.com/paper/2209.04161/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 50 references that are in the map.
- First posted on arXiv 9 Sep 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?](https://mlsanity.com/paper/1804.06323/) (2018)
- [Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization](https://mlsanity.com/paper/1710.05179/) (2017)
- [TFApprox: Towards a Fast Emulation of DNN Approximate Hardware Accelerators on GPU](https://mlsanity.com/paper/2002.09481/) (2020)
- [Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq](https://mlsanity.com/paper/1805.10387/) (2018)
- [The Effects of Approximate Multiplication on Convolutional Neural Networks](https://mlsanity.com/paper/2007.10500/) (2020)
- [Deep Learning Training with Simulated Approximate Multipliers](https://mlsanity.com/paper/2001.00060/) (2019)

## Built on by (most cited first, 9 of 9)

- [TransAxx: Efficient Transformers with Approximate Computing](https://mlsanity.com/paper/2402.07545/) (2024)
- [Approximate Computing Survey, Part II: Application-Specific & Architectural Approximation Techniques and Applications](https://mlsanity.com/paper/2307.11128/) (2023)
- [Training Neural Networks for Execution on Approximate Hardware](https://mlsanity.com/paper/2304.04125/) (2023)
- [FAMES: Fast Approximate Multiplier Substitution for Mixed-Precision Quantized DNNs--Down to 2 Bits!](https://mlsanity.com/paper/2411.18055/) (2024)
- [Carbon-Efficient 3D DNN Acceleration: Optimizing Performance and Sustainability](https://mlsanity.com/paper/2504.09851/) (2025)
- [Gradient Estimation Methods of Approximate Multipliers for High-Accuracy Retraining of Deep Learning Models](https://mlsanity.com/paper/2509.10519/) (2025)
- [Approximate Multiplier Induced Error Propagation in Deep Neural Networks](https://mlsanity.com/paper/2512.06537/) (2025)
- [AxMoE: Characterizing the Impact of Approximate Multipliers on Mixture-of-Experts DNN Architectures](https://mlsanity.com/paper/2605.04754/) (2026)
- [FAME: An FPGA-Based Platform for Approximate Multipliers Evaluation with Pattern-Guided DNN Retraining](https://mlsanity.com/paper/2609.17730/) (2026)

## Abstract (arXiv)

Edge training of Deep Neural Networks (DNNs) is a desirable goal for continuous learning; however, it is hindered by the enormous computational power required by training. Hardware approximate multipliers have shown their effectiveness for gaining resource-efficiency in DNN inference accelerators; however, training with approximate multipliers is largely unexplored. To build resource efficient accelerators with approximate multipliers supporting DNN training, a thorough evaluation of training convergence and accuracy for different DNN architectures and different approximate multipliers is needed. This paper presents ApproxTrain, an open-source framework that allows fast evaluation of DNN training and inference using simulated approximate multipliers. ApproxTrain is as user-friendly as TensorFlow (TF) and requires only a high-level description of a DNN architecture along with C/C++ functional models of the approximate multiplier. We improve the speed of the simulation at the multiplier level by using a novel LUT-based approximate floating-point (FP) multiplier simulator on GPU (AMSim). ApproxTrain leverages CUDA and efficiently integrates AMSim into the TensorFlow library, in order to overcome the absence of native hardware approximate multiplier in commercial GPUs. We use ApproxTrain to evaluate the convergence and accuracy of DNN training with approximate multipliers for small and large datasets (including ImageNet) using LeNets and ResNets architectures. The evaluations demonstrate similar convergence behavior and negligible change in test accuracy compared to FP32 and bfloat16 multipliers. Compared to CPU-based approximate multiplier simulations in training and inference, the GPU-accelerated ApproxTrain is more than 2500x faster. Based on highly optimized closed-source cuDNN/cuBLAS libraries with native hardware multipliers, the original TensorFlow is only 8x faster than ApproxTrain.