# Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead

arXiv: https://arxiv.org/abs/2012.11233 · HTML: https://mlsanity.com/paper/2012.11233/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2020.

## Built on by (most cited first, 30 of 34)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [Zeus: Understanding and Optimizing GPU Energy Consumption of DNN Training](https://mlsanity.com/paper/2208.06102/) (2022)
- [Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey](https://mlsanity.com/paper/2203.08737/) (2022)
- [A Survey on Quantum Machine Learning: Current Trends, Challenges, Opportunities, and the Road Ahead](https://mlsanity.com/paper/2310.10315/) (2023)
- [CarSNN: An Efficient Spiking Neural Network for Event-Based Autonomous Cars on the Loihi Neuromorphic Research Processor](https://mlsanity.com/paper/2107.00401/) (2021)
- [DVS-Attacks: Adversarial Attacks on Dynamic Vision Sensors for Spiking Neural Networks](https://mlsanity.com/paper/2107.00415/) (2021)
- [SwiftTron: An Efficient Hardware Accelerator for Quantized Transformers](https://mlsanity.com/paper/2304.03986/) (2023)
- [Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework](https://mlsanity.com/paper/2109.09829/) (2021)
- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey](https://mlsanity.com/paper/2405.00314/) (2024)
- [LaneSNNs: Spiking Neural Networks for Lane Detection on the Loihi Neuromorphic Processor](https://mlsanity.com/paper/2208.02253/) (2022)
- [R-SNN: An Analysis and Design Methodology for Robustifying Spiking Neural Networks against Adversarial Attacks through Noise Filters for Dynamic Vision Sensors](https://mlsanity.com/paper/2109.00533/) (2021)
- [fakeWeather: Adversarial Attacks for Deep Neural Networks Emulating Weather Conditions on the Camera Lens of Autonomous Systems](https://mlsanity.com/paper/2205.13807/) (2022)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systems](https://mlsanity.com/paper/2204.09514/) (2022)
- [Massively Digitized Power Grid: Opportunities and Challenges of Use-inspired AI](https://mlsanity.com/paper/2205.05180/) (2022)
- [RoHNAS: A Neural Architecture Search Framework with Conjoint Optimization for Adversarial Robustness and Hardware Efficiency of Convolutional and Capsule Networks](https://mlsanity.com/paper/2210.05276/) (2022)
- [Measurement-driven neural-network training for integrated magnetic tunnel junction arrays](https://mlsanity.com/paper/2312.06446/) (2023)
- [FAQ: Mitigating the Impact of Faults in the Weight Memory of DNN Accelerators through Fault-Aware Quantization](https://mlsanity.com/paper/2305.12590/) (2023)
- [Neural Network Methods for Radiation Detectors and Imaging](https://mlsanity.com/paper/2311.05726/) (2023)
- [Optimizing Dense Feed-Forward Neural Networks](https://mlsanity.com/paper/2312.10560/) (2023)
- [Robust ADAS: Enhancing Robustness of Machine Learning-based Advanced Driver Assistance Systems for Adverse Weather](https://mlsanity.com/paper/2407.02581/) (2024)
- [Implementation of a Binary Neural Network on a Passive Array of Magnetic Tunnel Junctions](https://mlsanity.com/paper/2112.09159/) (2021)
- [Operation-Level Performance Benchmarking of Graph Neural Networks for Scientific Applications](https://mlsanity.com/paper/2207.09955/) (2022)
- [RobCaps: Evaluating the Robustness of Capsule Networks against Affine Transformations and Adversarial Attacks](https://mlsanity.com/paper/2304.03973/) (2023)
- [HyperTuner: A Cross-Layer Multi-Objective Hyperparameter Auto-Tuning Framework for Data Analytic Services](https://mlsanity.com/paper/2304.10051/) (2023)
- [A Construction Kit for Efficient Low Power Neural Network Accelerator Designs](https://mlsanity.com/paper/2106.12810/) (2021)
- [Preventing Distillation-based Attacks on Neural Network IP](https://mlsanity.com/paper/2204.00292/) (2022)
- [Leveraging the HW/SW Optimizations and Ecosystems that Drive the AI Revolution](https://mlsanity.com/paper/2208.02808/) (2022)

## Abstract (arXiv)

Currently, Machine Learning (ML) is becoming ubiquitous in everyday life. Deep Learning (DL) is already present in many applications ranging from computer vision for medicine to autonomous driving of modern cars as well as other sectors in security, healthcare, and finance. However, to achieve impressive performance, these algorithms employ very deep networks, requiring a significant computational power, both during the training and inference time. A single inference of a DL model may require billions of multiply-and-accumulated operations, making the DL extremely compute- and energy-hungry. In a scenario where several sophisticated algorithms need to be executed with limited energy and low latency, the need for cost-effective hardware platforms capable of implementing energy-efficient DL execution arises. This paper first introduces the key properties of two brain-inspired models like Deep Neural Network (DNN), and Spiking Neural Network (SNN), and then analyzes techniques to produce efficient and high-performance designs. This work summarizes and compares the works for four leading platforms for the execution of algorithms such as CPU, GPU, FPGA and ASIC describing the main solutions of the state-of-the-art, giving much prominence to the last two solutions since they offer greater design flexibility and bear the potential of high energy-efficiency, especially for the inference process. In addition to hardware solutions, this paper discusses some of the important security issues that these DNN and SNN models may have during their execution, and offers a comprehensive section on benchmarking, explaining how to assess the quality of different networks and hardware systems designed for them.