mlsanityData to 14 Oct 2026

From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks

arXiv 2405.06038 ↗First posted 9 May 2024cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have been widely used in many artificial intelligence (AI) tasks. However, deploying them brings significant challenges due to the huge cost of memory, energy, and computation. To address these challenges, researchers have developed various model compression techniques such as model quantization and model pruning. Recently, there has been a surge in research of compression methods to achieve model efficiency while retaining the performance. Furthermore, more and more works focus on customizing the DNN hardware accelerators to better leverage the model compression techniques. In addition to efficiency, preserving security and privacy is critical for deploying DNNs. However, the vast and diverse body of related works can be overwhelming. This inspires us to conduct a comprehensive survey on recent research toward the goal of high-performance, cost-efficient, and safe deployment of DNNs. Our survey first covers the mainstream model compression techniques such as model quantization, model pruning, knowledge distillation, and optimizations of non-linear operations. We then introduce recent advances in designing hardware accelerators that can adapt to efficient model compression approaches. Additionally, we discuss how homomorphic encryption can be integrated to secure DNN deployment. Finally, we discuss several issues, such as hardware evaluation, generalization, and integration of various compression approaches. Overall, we aim to provide a big picture of efficient DNNs, from algorithm to hardware accelerators and security perspectives.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 199 of its 332 references that are in the map. First posted on arXiv 9 May 2024.

Builds on

199 of 332 references.

References · 199

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2016Densely Connected Convolutional Networkscited by 6,720
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  9. 2014Going Deeper with Convolutionscited by 4,241
  10. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  12. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
38 more
  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  2. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  3. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  4. 2020Knowledge Distillation: A Surveycited by 1,603
  5. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  6. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  7. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  8. 2019Relational Knowledge Distillationcited by 889
  9. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  10. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  11. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  12. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  13. 2016Sequence-Level Knowledge Distillationcited by 801
  14. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  15. 2019Contrastive Representation Distillationcited by 748
  16. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  17. 2017Deep Mutual Learningcited by 628
  18. 2019Learned Step Size Quantizationcited by 628
  19. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  20. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  21. 2018Born Again Neural Networkscited by 546
  22. 2019Importance Estimation for Neural Network Pruningcited by 541
  23. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  24. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  25. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  26. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  27. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  28. 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 434
  29. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  30. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  31. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  32. 2022Decoupled Knowledge Distillationcited by 388
  33. 2017Soft-NMS -- Improving Object Detection With One Line of Codecited by 382
  34. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  35. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 367
  36. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  37. 2019Variational Information Distillation for Knowledge Transfercited by 324
  38. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322

50 most cited of 199 All 199 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deploymentcited by 6
  2. 2025Hardware-Algorithm Co-Optimization of Early-Exit Neural Networks for Multi-Core Edge Acceleratorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization
  2. 2026Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks
  3. 2021PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation
  4. 2022Characterizing and Understanding the Behavior of Quantized Models for Reliable Deployment
  5. 2022Towards Hardware-Specific Automatic Compression of Neural Networks
  6. 2026Kilobyte Models: Neural Networks as a Seed and a Quantized Latent
  7. 2024Comprehensive Study on Performance Evaluation and Optimization of Model Compression: Bridging Traditional Deep Learning and Large Language Models
  8. 2022Energy-efficient Deployment of Deep Learning Applications on Cortex-M based Microcontrollers using Deep Compression