mlsanityData to 14 Oct 2026

Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey

arXiv 2203.08737 ↗First posted 16 Mar 2022cs.AR cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

Deep Neural Networks (DNNs) are very popular because of their high performance in various cognitive tasks in Machine Learning (ML). Recent advancements in DNNs have brought beyond human accuracy in many tasks, but at the cost of high computational complexity. To enable efficient execution of DNN inference, more and more research works, therefore, exploit the inherent error resilience of DNNs and employ Approximate Computing (AC) principles to address the elevated energy demands of DNN accelerators. This article provides a comprehensive survey and analysis of hardware approximation techniques for DNN accelerators. First, we analyze the state of the art and by identifying approximation families, we cluster the respective works with respect to the approximation type. Next, we analyze the complexity of the performed evaluations (with respect to the dataset and DNN size) to assess the efficiency, the potential, and limitations of approximate DNN accelerators. Moreover, a broad discussion is provided, regarding error metrics that are more suitable for designing approximate units for DNN accelerators as well as accuracy recovery approaches that are tailored to DNN inference. Finally, we present how Approximate Computing for DNN accelerators can go beyond energy efficiency and address reliability and security issues, as well.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 47 of its 143 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

47 of 143 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  5. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  6. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  7. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  8. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  9. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  10. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  11. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  12. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
35 more
  1. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 434
  2. 2015Deep Learning with Limited Numerical Precisioncited by 376
  3. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 295
  4. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  5. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  6. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 254
  7. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  8. 2016Trained Ternary Quantizationcited by 208
  9. 2019The Evolved Transformercited by 180
  10. 2018Training Deep Neural Networks with 8-bit Floating Point Numberscited by 164
  11. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  12. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 135
  13. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 109
  14. 2018The Best of Both Worlds: Combining Recent Advances in Neural Machine Translationcited by 102
  15. 2015Neural Networks with Few Multiplicationscited by 48
  16. 2021Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Aheadcited by 41
  17. 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 39
  18. 2020Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Aheadcited by 34
  19. 2018XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inferencecited by 30
  20. 2017Attacking Binarized Neural Networkscited by 29
  21. 2019ALWANN: Automatic Layer-Wise Approximation of Deep Neural Network Accelerators without Retrainingcited by 28
  22. 2019EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAMcited by 24
  23. 2018BISMO: A Scalable Bit-Serial Matrix Multiplication Overlay for Reconfigurable Computingcited by 22
  24. 2018QuSecNets: Quantization-based Defense Mechanism for Securing Deep Neural Network against Adversarial Attackscited by 21
  25. 2020TFApprox: Towards a Fast Emulation of DNN Approximate Hardware Accelerators on GPUcited by 17
  26. 2021Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-designcited by 13
  27. 2017Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networkscited by 12
  28. 2021Control Variate Approximation for DNN Acceleratorscited by 11
  29. 2019Discretization based Solutions for Secure Machine Learning against Adversarial Attackscited by 10
  30. 2018AxTrain: Hardware-Oriented Neural Network Training for Approximate Inferencecited by 8
  31. 2016YodaNN: An Architecture for Ultra-Low Power Binary-Weight CNN Accelerationcited by 7
  32. 2018XNORBIN: A 95 TOp/s/W Hardware Accelerator for Binary Convolutional Neural Networkscited by 5
  33. 2019Automated Circuit Approximation Method Driven by Data Distributioncited by 5
  34. 2019AddNet: Deep Neural Networks Using FPGA-Optimized Multiplierscited by 5
  35. 2021Reliability-Aware Quantization for Anti-Aging NPUscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 6; 2024: 14; 2025: 10; 2026: 5.

39 papers.

Sort:

By area:

  1. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 80
  2. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
  3. 2024Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inferencecited by 61
  4. 2023Federated Learning for Computationally-Constrained Heterogeneous Devices: A Surveycited by 40
  5. 2025LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Acceleratorcited by 9
  6. 2022Approximate Computing and the Efficient Machine Learning Expeditioncited by 8
  7. 2023Model-to-Circuit Cross-Approximation For Printed Machine Learning Classifierscited by 7
  8. 2024TransAxx: Efficient Transformers with Approximate Computingcited by 7
  9. 2025DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantizationcited by 6
  10. 2023Special Session: Approximation and Fault Resiliency of DNN Acceleratorscited by 5
  11. 2024Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operationscited by 5
  12. 2023Bespoke Approximation of Multiplication-Accumulation and Activation Targeting Printed Multilayer Perceptronscited by 4
27 more
  1. 2024HOAA: Hybrid Overestimating Approximate Adder for Enhanced Performance Processing Enginecited by 4
  2. 2023Approximate Computing Survey, Part II: Application-Specific & Architectural Approximation Techniques and Applicationscited by 3
  3. 2024Embedding Hardware Approximations in Discrete Genetic-based Training for Printed MLPscited by 3
  4. 2024Recall: Empowering Multimodal Embedding for Edge Devicescited by 3
  5. 2025Green Prompting: Characterizing Prompt-driven Energy Costs of LLM Inferencecited by 3
  6. 2024AdAM: Adaptive Fault-Tolerant Approximate Multiplier for Edge DNN Acceleratorscited by 2
  7. 2022Computational Complexity Evaluation of Neural Network Applications in Signal Processingcited by 1
  8. 2024ApproxDARTS: Differentiable Neural Architecture Search with Approximate Multiplierscited by 1
  9. 2024PACiM: A Sparsity-Centric Hybrid Compute-in-Memory Architecture via Probabilistic Approximationcited by 1
  10. 2024Leveraging Highly Approximated Multipliers in DNN Inferencecited by 1
  11. 2026MCEL: Margin-Based Cross-Entropy Loss for Error-Tolerant Quantized Neural Networkscited by 1
  12. 2022Energy-efficient DNN Inference on Approximate Accelerators Through Formal Property Explorationcited by 0
  13. 2024EncodingNet: A Novel Encoding-based MAC Design for Efficient Neural Network Accelerationcited by 0
  14. 2024Exploring Quantization and Mapping Synergy in Hardware-Aware Deep Neural Network Acceleratorscited by 0
  15. 2024Exploring DNN Robustness Against Adversarial Attacks Using Approximate Multiplierscited by 0
  16. 2024Accelerating TinyML Inference on Microcontrollers through Approximate Kernelscited by 0
  17. 2025Explainable AI-Guided Efficient Approximate DNN Generation for Multi-Pod Systolic Arrayscited by 0
  18. 2025ConvShareViT: Enhancing Vision Transformers with Convolutional Attention Mechanisms for Free-Space Optical Acceleratorscited by 0
  19. 2025BitParticle: Partializing Sparse Dual-Factors to Build Quasi-Synchronizing MAC Arrays for Energy-efficient DNNscited by 0
  20. 2025Arbitrary Precision Printed Ternary Neural Networks with Holistic Evolutionary Approximationcited by 0
  21. 2025Gradient Estimation Methods of Approximate Multipliers for High-Accuracy Retraining of Deep Learning Modelscited by 0
  22. 2025MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Accelerationcited by 0
  23. 2025Approximate Multiplier Induced Error Propagation in Deep Neural Networkscited by 0
  24. 2026'1'-bit Count-based Sorting Unit to Reduce Link Power in DNN Acceleratorscited by 0
  25. 2026OpenACMv2: An Accuracy-Constrained Co-Optimization Framework for Approximate DCiMcited by 0
  26. 2026Genetic Programming with Transformer-Based Mutation for Approximate Circuit Designcited by 0
  27. 2026Toward Energy-Efficient and Low-Power Arrhythmia Detection for Wearable Devicescited by 0