RCT: Resource Constrained Training for Edge AI
Not on the map. Why?
Abstract (arXiv)
Neural networks training on edge terminals is essential for edge AI computing, which needs to be adaptive to evolving environment. Quantised models can efficiently run on edge devices, but existing training methods for these compact models are designed to run on powerful servers with abundant memory and energy budget. For example, quantisation-aware training (QAT) method involves two copies of model parameters, which is usually beyond the capacity of on-chip memory in edge devices. Data movement between off-chip and on-chip memory is energy demanding as well. The resource requirements are trivial for powerful servers, but critical for edge devices. To mitigate these issues, We propose Resource Constrained Training (RCT). RCT only keeps a quantised model throughout the training, so that the memory requirements for model parameters in training is reduced. It adjusts per-layer bitwidth dynamically in order to save energy when a model can learn effectively with lower precision. We carry out experiments with representative models and tasks in image application and natural language processing. Experiments show that RCT saves more than 86% energy for General Matrix Multiply (GEMM) and saves more than 46% memory for model parameters, with limited accuracy loss. Comparing with QAT-based method, RCT saves about half of energy on moving model parameters.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 55 of its 84 references that are in the map. First posted on arXiv 26 Mar 2021.
Builds on
55 of 84 references.
References · 55
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
- 2016Wide Residual Networkscited by 3,057
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
38 more
- 2017Mixed Precision Trainingcited by 1,135
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
- 2018Rethinking the Value of Network Pruningcited by 595
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
- 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
- 2019The State of Sparsity in Deep Neural Networkscited by 471
- 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
- 2015Deep Learning with Limited Numerical Precisioncited by 368
- 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
- 2019Q8BERT: Quantized 8Bit BERTcited by 296
- 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
- 2021I-BERT: Integer-only BERT Quantizationcited by 250
- 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
- 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 210
- 2016Ternary Weight Networkscited by 195
- 2018Composition Loss for Counting, Density Map Estimation and Localization in Dense Crowdscited by 186
- 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
- 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 175
- 2019Quantization Networkscited by 151
- 2018Scalable Methods for 8-bit Training of Neural Networkscited by 139
- 2018Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Searchcited by 122
- 2020GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inferencecited by 103
- 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 95
- 2018Training and Inference with Integers in Deep Neural Networkscited by 84
- 2020Post-Training Piecewise Linear Quantization for Deep Neural Networkscited by 77
- 2019Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detectioncited by 73
- 2017Adaptive Quantization for Deep Neural Networkcited by 67
- 2020Robust Quantization: One Model to Rule Them Allcited by 63
- 2019Low-Memory Neural Network Training: A Technical Reportcited by 56
- 2020CNN-based Density Estimation and Crowd Counting: A Surveycited by 43
- 2019Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integerscited by 32
- 2018NICE: Noise Injection and Clamping Estimation for Neural Network Quantizationcited by 24
- 2020BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activationscited by 23
- 2018BISMO: A Scalable Bit-Serial Matrix Multiplication Overlay for Reconfigurable Computingcited by 21
- 2017A Survey of FPGA-Based Neural Network Acceleratorcited by 17
- 2019Towards Efficient Training for Neural Network Quantizationcited by 16
- 2017Ternary Neural Networks with Fine-Grained Quantizationcited by 15
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2023: 1.
1 paper.