AdaSpring: Context-adaptive and Runtime-evolutionary Deep Model Compression for Mobile Applications
Not on the map. Why?
Abstract (arXiv)
There are many deep learning (e.g., DNN) powered mobile and wearable applications today continuously and unobtrusively sensing the ambient surroundings to enhance all aspects of human lives. To enable robust and private mobile sensing, DNN tends to be deployed locally on the resource-constrained mobile devices via model compression. The current practice either hand-crafted DNN compression techniques, i.e., for optimizing DNN-relative performance (e.g., parameter size), or on-demand DNN compression methods, i.e., for optimizing hardware-dependent metrics (e.g., latency), cannot be locally online because they require offline retraining to ensure accuracy. Also, none of them have correlated their efforts with runtime adaptive compression to consider the dynamic nature of the deployment context of mobile applications. To address those challenges, we present AdaSpring, a context-adaptive and self-evolutionary DNN compression framework. It enables the runtime adaptive DNN compression locally online. Specifically, it presents the ensemble training of a retraining-free and self-evolutionary network to integrate multiple alternative DNN compression configurations (i.e., compressed architectures and weights). It then introduces the runtime search strategy to quickly search for the most suitable compression configurations and evolve the corresponding weights. With evaluation on five tasks across three platforms and a real-world case study, experiment outcomes show that AdaSpring obtains up to 3.1x latency reduction, 4.2 x energy efficiency improvement in DNNs, compared to hand-crafted compression techniques, while only incurring <= 6.2ms runtime-evolution latency.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 62 references that are in the map. First posted on arXiv 28 Jan 2021.
Builds on
36 of 62 references.
References · 36
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
- 2018DARTS: Differentiable Architecture Searchcited by 1,798
- 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
- 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
- 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
- 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
- 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
- 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
24 more
- 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
- 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 301
- 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 254
- 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
- 2018Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantificationcited by 199
- 2017Hierarchical Representations for Efficient Architecture Searchcited by 191
- 2017BlockDrop: Dynamic Inference Paths in Residual Networkscited by 189
- 2016Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruningcited by 134
- 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
- 2019Convergence of Edge Computing and Deep Learning: A Comprehensive Surveycited by 120
- 2018AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inferencecited by 77
- 2018ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptationcited by 76
- 2018SqueezeNext: Hardware-Aware Neural Network Designcited by 61
- 2019DABNet: Depth-wise Asymmetric Bottleneck for Real-time Semantic Segmentationcited by 45
- 2018Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutionscited by 33
- 2019AutoDispNet: Improving Disparity Estimation With AutoMLcited by 29
- 2015Compressing Convolutional Neural Networkscited by 25
- 2020Theory-Inspired Path-Regularized Differential Network Architecture Searchcited by 25
- 2017DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Frameworkcited by 22
- 2020Fast Neural Network Adaptation via Parameter Remapping and Architecture Searchcited by 16
- 2019Play and Prune: Adaptive Filter Pruning for Deep Model Compressioncited by 15
- 2020AdaDeep: A Usage-Driven, Automated Deep Model Compression Framework for Enabling Ubiquitous Intelligent Mobilescited by 10
- 2020The Ramifications of Making Deep Neural Networks Compactcited by 6
- 2020Modeling Data Reuse in Deep Neural Networks by Taking Data-Types into Cognizancecited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.