Colab NAS: Obtaining lightweight task-specific convolutional neural networks following Occam's razor
Not on the map. Why?
Abstract (arXiv)
The current trend of applying transfer learning from convolutional neural networks (CNNs) trained on large datasets can be an overkill when the target application is a custom and delimited problem, with enough data to train a network from scratch. On the other hand, the training of custom and lighter CNNs requires expertise, in the from-scratch case, and or high-end resources, as in the case of hardware-aware neural architecture search (HW NAS), limiting access to the technology by non-habitual NN developers. For this reason, we present ColabNAS, an affordable HW NAS technique for producing lightweight task-specific CNNs. Its novel derivative-free search strategy, inspired by Occam's razor, allows to obtain state-of-the-art results on the Visual Wake Word dataset, a standard TinyML benchmark, in just 3.1 GPU hours using free online GPU services such as Google Colaboratory and Kaggle Kernel.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 43 references that are in the map. First posted on arXiv 15 Dec 2022.
Builds on
27 of 43 references.
References · 27
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
- 2019Searching for MobileNetV3cited by 1,984
- 2018DARTS: Differentiable Architecture Searchcited by 1,782
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
- 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
15 more
- 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
- 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
- 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
- 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
- 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
- 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 269
- 2020Benchmarking TinyML Systems: Challenges and Directioncited by 89
- 2021MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learningcited by 87
- 2022A Review of Deep Transfer Learning and Recent Advancementscited by 56
- 2020Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Taskscited by 43
- 2020Best of Both Worlds: AutoML Codesign of a CNN and its Hardware Acceleratorcited by 30
- 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
- 2020NASCaps: A Framework for Neural Architecture Search to Optimize the Accuracy and Hardware Efficiency of Convolutional Capsule Networkscited by 16
- 2019DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruningcited by 13
- 2021WheatNet: A Lightweight Convolutional Neural Network for High-throughput Image-based Wheat Head Detection and Countingcited by 8
Built on by
Citing papers per year: 2024: 1.
1 paper.