3U-EdgeAI: Ultra-Low Memory Training, Ultra-Low BitwidthQuantization, and Ultra-Low Latency Acceleration
Not on the map. Why?
Abstract (arXiv)
The deep neural network (DNN) based AI applications on the edge require both low-cost computing platforms and high-quality services. However, the limited memory, computing resources, and power budget of the edge devices constrain the effectiveness of the DNN algorithms. Developing edge-oriented AI algorithms and implementations (e.g., accelerators) is challenging. In this paper, we summarize our recent efforts for efficient on-device AI development from three aspects, including both training and inference. First, we present on-device training with ultra-low memory usage. We propose a novel rank-adaptive tensor-based tensorized neural network model, which offers orders-of-magnitude memory reduction during training. Second, we introduce an ultra-low bitwidth quantization method for DNN model compression, achieving the state-of-the-art accuracy under the same compression ratio. Third, we introduce an ultra-low latency DNN accelerator design, practicing the software/hardware co-design methodology. This paper emphasizes the importance and efficacy of training, quantization and accelerator design, and calls for more research breakthroughs in the area for AI on the edge.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 25 of its 43 references that are in the map. First posted on arXiv 11 May 2021.
Builds on
25 of 43 references.
References · 25
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
- 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
- 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
- 2015Tensorizing Neural Networkscited by 246
- 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
- 2016Ternary Weight Networkscited by 195
- 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 141
- 2017Distributed Deep Neural Networks over the Cloud, the Edge and End Devicescited by 119
- 2017Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMMcited by 60
- 2016Ultimate tensorization: compressing convolutional and FC layers alikecited by 55
13 more
- 2019FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edgecited by 48
- 2019E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savingscited by 36
- 2016Ternary Neural Networks for Resource-Efficient AI Applicationscited by 30
- 2017Compressing Recurrent Neural Network with Tensor Traincited by 28
- 2019Tensorized Embedding Layers for Efficient Model Compressioncited by 27
- 2019Bayesian Tensorized Neural Networks with Automatic Rank Selectioncited by 24
- 2016Hardware-oriented Approximation of Convolutional Neural Networkscited by 20
- 2020Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determinationcited by 17
- 2020VecQ: Minimal Loss DNN Model Compression With Vectorized Weight Quantizationcited by 15
- 2021Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-designcited by 13
- 2020Compressing Recurrent Neural Networks Using Hierarchical Tucker Tensor Decompositioncited by 9
- 2021On-FPGA Training with Ultra Memory Reduction: A Low-Precision Tensor Methodcited by 9
- 2016FPGA Based Implementation of Deep Neural Networks Using On-chip Memory Onlycited by 2
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 1.
3 papers.