Computational Complexity Evaluation of Neural Network Applications in Signal Processing
Not on the map. Why?
Abstract (arXiv)
In this paper, we provide a systematic approach for assessing and comparing the computational complexity of neural network layers in digital signal processing. We provide and link four software-to-hardware complexity measures, defining how the different complexity metrics relate to the layers' hyper-parameters. This paper explains how to compute these four metrics for feed-forward and recurrent layers, and defines in which case we ought to use a particular metric depending on whether we characterize a more soft- or hardware-oriented application. One of the four metrics, called `the number of additions and bit shifts (NABS)', is newly introduced for heterogeneous quantization. NABS characterizes the impact of not only the bitwidth used in the operation but also the type of quantization used in the arithmetical operations. We intend this work to serve as a baseline for the different levels (purposes) of complexity estimation related to the neural networks' application in real-time digital signal processing, aiming at unifying the computational complexity estimation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 40 of its 119 references that are in the map. First posted on arXiv 24 Jun 2022.
Builds on
40 of 119 references.
References · 40
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2016Layer Normalizationcited by 5,579
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
- 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
- 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 427
- 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
- 2017Gate-Variants of Gated Recurrent Unit (GRU) Neural Networkscited by 239
- 2015A Critical Review of Recurrent Neural Networks for Sequence Learningcited by 223
- 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
28 more
- 20191D Convolutional Neural Networks and Applications: A Surveycited by 152
- 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
- 2018Scalable Methods for 8-bit Training of Neural Networkscited by 139
- 2016Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruningcited by 133
- 2020Bayesian Bits: Unifying Quantization and Pruningcited by 93
- 2019DeepShift: Towards Multiplication-Less Neural Networkscited by 52
- 2020ShiftAddNet: A Hardware-Inspired Deep Networkcited by 52
- 2019Any-Precision Deep Neural Networkscited by 39
- 2022Hardware Approximate Techniques for Deep Neural Network Accelerators: A Surveycited by 39
- 2018Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutionscited by 33
- 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
- 2022Power-of-Two Quantization for Low Bitwidth and Hardware Compliant Neural Networkscited by 26
- 2016Hardware-oriented Approximation of Convolutional Neural Networkscited by 20
- 2021Performance versus Complexity Study of Neural Network Equalizers in Coherent Optical Systemscited by 17
- 2017Deep Learning with Dynamic Computation Graphscited by 15
- 2018Compact and Computationally Efficient Representation of Deep Neural Networkscited by 11
- 2020A Review of Designs and Applications of Echo State Networkscited by 11
- 2018On the Statistical Challenges of Echo State Networks and Some Potential Remediescited by 7
- 2021Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platformscited by 7
- 2021Transfer Learning for Neural Networks-based Equalizers in Coherent Optical Systemscited by 7
- 2021A Survey and Empirical Evaluation of Parallel Deep Learning Frameworkscited by 7
- 2021Performance and Complexity Analysis of bi-directional Recurrent Neural Network Models vs. Volterra Nonlinear Equalizers in Digital Coherent Systemscited by 6
- 2021Neural Networks-based Equalizers for Coherent Optical Transmission: Caveats and Pitfallscited by 5
- 2022Domain Adaptation: the Key Enabler of Neural Network Equalizers in Coherent Optical Systemscited by 4
- 2021Experimental implementation of a neural network optical channel equalizer in restricted hardware using pruning and quantizationcited by 3
- 2019AdaNN: Adaptive Neural Network-based Equalizer via Online Semi-supervised Learningcited by 2
- 2019Revisiting Multi-Step Nonlinearity Compensation with Machine Learningcited by 1
- 2020MSP: An FPGA-Specific Mixed-Scheme, Multi-Precision Deep Neural Network Quantization Frameworkcited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.