mlsanityData to 7 Oct 2026

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

arXiv 1703.09039 ↗First posted 27 Mar 2017cs.CVcited by 427

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances towards the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic co-designs, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the trade-offs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Key facts. Cited by 427 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 87; 2022: 58; 2023: 77; 2024: 75; 2025: 73; 2026: 57.

427 papers.

Sort:

By area:

  1. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  2. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  3. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  4. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  5. 2023A Comprehensive Survey of Dataset Distillationcited by 114
  6. 2022Attention Spiking Neural Networkscited by 103
  7. 2021Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applicationscited by 96
  8. 2021Trustworthy AI: A Computational Perspectivecited by 93
  9. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  10. 2024A Survey on Kolmogorov-Arnold Networkcited by 67
  11. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  12. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 57
38 more
  1. 2021Beyond Classification: Directly Training Spiking Neural Networks for Semantic Segmentationcited by 54
  2. 2021A Survey of Machine Learning for Computer Architecture and Systemscited by 53
  3. 2023Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based acceleratorscited by 42
  4. 2021Recent Advances on Neural Network Pruning at Initializationcited by 41
  5. 2021Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Aheadcited by 40
  6. 2022Hardware Approximate Techniques for Deep Neural Network Accelerators: A Surveycited by 39
  7. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 38
  8. 2021A TinyML Platform for On-Device Continual Learning with Quantized Latent Replayscited by 37
  9. 2023TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterestcited by 35
  10. 2024Machine Unlearning: A Comprehensive Surveycited by 33
  11. 2022DL4SciVis: A State-of-the-Art Survey on Deep Learning for Scientific Visualizationcited by 32
  12. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  13. 2021An optical neural network using less than 1 photon per multiplicationcited by 30
  14. 2022Energy-Latency Attacks via Sponge Poisoningcited by 29
  15. 2025Vision-Language Models for Edge Networks: A Comprehensive Surveycited by 29
  16. 2021CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparabilitycited by 28
  17. 2021CarSNN: An Efficient Spiking Neural Network for Event-Based Autonomous Cars on the Loihi Neuromorphic Research Processorcited by 27
  18. 2021Visual Explanations from Spiking Neural Networks using Interspike Intervalscited by 26
  19. 2021Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecturecited by 25
  20. 2022HALSIE: Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalitiescited by 24
  21. 2023Green Edge AI: A Contemporary Surveycited by 24
  22. 2023A Systematic Literature Review on Hardware Reliability Assessment Methods for Deep Neural Networkscited by 23
  23. 2023Binarized Spectral Compressive Imagingcited by 23
  24. 2023Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithmcited by 23
  25. 2023Reducing Energy Bloat in Large Model Trainingcited by 22
  26. 2024HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shiftscited by 22
  27. 2021Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligencecited by 21
  28. 2023A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platformscited by 21
  29. 2024Towards Trustworthy AI: A Review of Ethical and Robust Large Language Modelscited by 21
  30. 2024Watermarking Techniques for Large Language Models: A Surveycited by 20
  31. 2022Spiking Neural Networks for Frame-based and Event-based Single Object Localizationcited by 18
  32. 2022Benchmarking simulated and physical quantum processing units using quantum and hybrid algorithmscited by 18
  33. 2021DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Devicecited by 17
  34. 2021Uncertainty Modeling of Emerging Device-based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Searchcited by 17
  35. 2021AI Accelerator Survey and Trendscited by 17
  36. 2021Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Frameworkcited by 17
  37. 2022SWIM: Selective Write-Verify for Computing-in-Memory Neural Acceleratorscited by 16
  38. 2022Trainability Preserving Neural Pruningcited by 16

50 most cited of 427 All 427 (JSON)