NetBooster: Empowering Tiny Deep Learning By Standing on the Shoulders of Deep Giants
Not on the map. Why?
Abstract (arXiv)
Tiny deep learning has attracted increasing attention driven by the substantial demand for deploying deep learning on numerous intelligent Internet-of-Things devices. However, it is still challenging to unleash tiny deep learning's full potential on both large-scale datasets and downstream tasks due to the under-fitting issues caused by the limited model capacity of tiny neural networks (TNNs). To this end, we propose a framework called NetBooster to empower tiny deep learning by augmenting the architectures of TNNs via an expansion-then-contraction strategy. Extensive experiments show that NetBooster consistently outperforms state-of-the-art tiny deep learning solutions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 23 Jun 2023.
Builds on
22 of 31 references.
References · 22
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
- 2019Contrastive Representation Distillationcited by 734
- 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
- 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
- 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
- 2019Improved Knowledge Distillation via Teacher Assistantcited by 567
- 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
- 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
- 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
10 more
- 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 269
- 2018DropBlock: A regularization method for convolutional networkscited by 261
- 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
- 2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 107
- 2019Knowledge Distillation via Route Constrained Optimizationcited by 73
- 2021DeepReDuce: ReLU Reduction for Fast Private Inferencecited by 50
- 2017Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Netcited by 27
- 2020Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Networkcited by 24
- 2021Network Augmentation for Tiny Deep Learningcited by 23
- 2021MIA-Former: Efficient and Robust Vision Transformers via Multi-grained Input-Adaptationcited by 12
Built on by
Citing papers per year: 2025: 1.
1 paper.