# Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey

arXiv: https://arxiv.org/abs/2203.08737 · HTML: https://mlsanity.com/paper/2203.08737/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 143 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 30 of 47)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions](https://mlsanity.com/paper/2006.02903/) (2020)
- [Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights](https://mlsanity.com/paper/1702.03044/) (2017)
- [Trained Ternary Quantization](https://mlsanity.com/paper/1612.01064/) (2016)
- [The Evolved Transformer](https://mlsanity.com/paper/1901.11117/) (2019)
- [Training Deep Neural Networks with 8-bit Floating Point Numbers](https://mlsanity.com/paper/1812.08011/) (2018)
- [Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss](https://mlsanity.com/paper/1808.05779/) (2018)
- [SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks](https://mlsanity.com/paper/1708.04485/) (2017)
- [Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks](https://mlsanity.com/paper/1712.01507/) (2017)
- [The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation](https://mlsanity.com/paper/1804.09849/) (2018)
- [Neural Networks with Few Multiplications](https://mlsanity.com/paper/1510.03009/) (2015)
- [Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead](https://mlsanity.com/paper/2101.02559/) (2021)
- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead](https://mlsanity.com/paper/2012.11233/) (2020)

## Built on by (most cited first, 30 of 39)

- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference](https://mlsanity.com/paper/2403.20306/) (2024)
- [Federated Learning for Computationally-Constrained Heterogeneous Devices: A Survey](https://mlsanity.com/paper/2307.09182/) (2023)
- [LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator](https://mlsanity.com/paper/2501.10658/) (2025)
- [Approximate Computing and the Efficient Machine Learning Expedition](https://mlsanity.com/paper/2210.00497/) (2022)
- [Model-to-Circuit Cross-Approximation For Printed Machine Learning Classifiers](https://mlsanity.com/paper/2303.08255/) (2023)
- [TransAxx: Efficient Transformers with Approximate Computing](https://mlsanity.com/paper/2402.07545/) (2024)
- [DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantization](https://mlsanity.com/paper/2507.12933/) (2025)
- [Special Session: Approximation and Fault Resiliency of DNN Accelerators](https://mlsanity.com/paper/2306.04645/) (2023)
- [Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operations](https://mlsanity.com/paper/2407.14274/) (2024)
- [Bespoke Approximation of Multiplication-Accumulation and Activation Targeting Printed Multilayer Perceptrons](https://mlsanity.com/paper/2312.17612/) (2023)
- [HOAA: Hybrid Overestimating Approximate Adder for Enhanced Performance Processing Engine](https://mlsanity.com/paper/2408.00806/) (2024)
- [Approximate Computing Survey, Part II: Application-Specific & Architectural Approximation Techniques and Applications](https://mlsanity.com/paper/2307.11128/) (2023)
- [Embedding Hardware Approximations in Discrete Genetic-based Training for Printed MLPs](https://mlsanity.com/paper/2402.02930/) (2024)
- [Recall: Empowering Multimodal Embedding for Edge Devices](https://mlsanity.com/paper/2409.15342/) (2024)
- [Green Prompting: Characterizing Prompt-driven Energy Costs of LLM Inference](https://mlsanity.com/paper/2503.10666/) (2025)
- [AdAM: Adaptive Fault-Tolerant Approximate Multiplier for Edge DNN Accelerators](https://mlsanity.com/paper/2403.02936/) (2024)
- [Computational Complexity Evaluation of Neural Network Applications in Signal Processing](https://mlsanity.com/paper/2206.12191/) (2022)
- [ApproxDARTS: Differentiable Neural Architecture Search with Approximate Multipliers](https://mlsanity.com/paper/2404.08002/) (2024)
- [PACiM: A Sparsity-Centric Hybrid Compute-in-Memory Architecture via Probabilistic Approximation](https://mlsanity.com/paper/2408.16246/) (2024)
- [Leveraging Highly Approximated Multipliers in DNN Inference](https://mlsanity.com/paper/2412.16757/) (2024)
- [MCEL: Margin-Based Cross-Entropy Loss for Error-Tolerant Quantized Neural Networks](https://mlsanity.com/paper/2603.05048/) (2026)
- [Energy-efficient DNN Inference on Approximate Accelerators Through Formal Property Exploration](https://mlsanity.com/paper/2207.12350/) (2022)
- [EncodingNet: A Novel Encoding-based MAC Design for Efficient Neural Network Acceleration](https://mlsanity.com/paper/2402.18595/) (2024)
- [Exploring Quantization and Mapping Synergy in Hardware-Aware Deep Neural Network Accelerators](https://mlsanity.com/paper/2404.05368/) (2024)
- [Exploring DNN Robustness Against Adversarial Attacks Using Approximate Multipliers](https://mlsanity.com/paper/2404.11665/) (2024)
- [Accelerating TinyML Inference on Microcontrollers through Approximate Kernels](https://mlsanity.com/paper/2409.16815/) (2024)
- [Explainable AI-Guided Efficient Approximate DNN Generation for Multi-Pod Systolic Arrays](https://mlsanity.com/paper/2503.16583/) (2025)
- [ConvShareViT: Enhancing Vision Transformers with Convolutional Attention Mechanisms for Free-Space Optical Accelerators](https://mlsanity.com/paper/2504.11517/) (2025)

## Abstract (arXiv)

Deep Neural Networks (DNNs) are very popular because of their high performance in various cognitive tasks in Machine Learning (ML). Recent advancements in DNNs have brought beyond human accuracy in many tasks, but at the cost of high computational complexity. To enable efficient execution of DNN inference, more and more research works, therefore, exploit the inherent error resilience of DNNs and employ Approximate Computing (AC) principles to address the elevated energy demands of DNN accelerators. This article provides a comprehensive survey and analysis of hardware approximation techniques for DNN accelerators. First, we analyze the state of the art and by identifying approximation families, we cluster the respective works with respect to the approximation type. Next, we analyze the complexity of the performed evaluations (with respect to the dataset and DNN size) to assess the efficiency, the potential, and limitations of approximate DNN accelerators. Moreover, a broad discussion is provided, regarding error metrics that are more suitable for designing approximate units for DNN accelerators as well as accuracy recovery approaches that are tailored to DNN inference. Finally, we present how Approximate Computing for DNN accelerators can go beyond energy efficiency and address reliability and security issues, as well.