QADAM: Quantization-Aware DNN Accelerator Modeling for Pareto-Optimality
Not on the map. Why?
Abstract (arXiv)
As the machine learning and systems communities strive to achieve higher energy-efficiency through custom deep neural network (DNN) accelerators, varied bit precision or quantization levels, there is a need for design space exploration frameworks that incorporate quantization-aware processing elements (PE) into the accelerator design space while having accurate and fast power, performance, and area models. In this work, we present QADAM, a highly parameterized quantization-aware power, performance, and area modeling framework for DNN accelerators. Our framework can facilitate future research on design space exploration and Pareto-efficiency of DNN accelerators for various design choices such as bit precision, PE type, scratchpad sizes of PEs, global buffer size, number of total PEs, and DNN configurations. Our results show that different bit precisions and PE types lead to significant differences in terms of performance per area and energy. Specifically, our framework identifies a wide range of design points where performance per area and energy varies more than 5x and 35x, respectively. We also show that the proposed lightweight processing elements (LightPEs) consistently achieve Pareto-optimal results in terms of accuracy and hardware-efficiency. With the proposed framework, we show that LightPEs achieve on par accuracy results and up to 5.7x more performance per area and energy improvement when compared to the best INT16 based design.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 34 references that are in the map. First posted on arXiv 20 May 2022.
Builds on
20 of 34 references.
References · 20
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
- 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
- 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
- 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
- 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
- 2019Towards Efficient Model Compression via Learned Global Rankingcited by 75
- 2020Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Taskscited by 43
8 more
- 2017NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networkscited by 30
- 2020Accelerator-aware Neural Network Design using AutoMLcited by 30
- 2018SCALE-Sim: Systolic CNN Accelerator Simulatorcited by 26
- 2021Rethinking Co-design of Neural Architectures and Hardware Acceleratorscited by 12
- 2018Hardware-Aware Machine Learning: Modeling and Optimizationcited by 11
- 2020The Architectural Implications of Distributed Reinforcement Learning on CPU-GPU Systemscited by 5
- 2020DeepNVM++: Cross-Layer Modeling and Optimization Framework of Non-Volatile Memories for Deep Learningcited by 4
- 2022QAPPA: Quantization-Aware Power, Performance, and Area Modeling of DNN Acceleratorscited by 4
Built on by
Citing papers per year: 2022: 2; 2023: 1.
3 papers.