# On Neural Architecture Search for Resource-Constrained Hardware Platforms

arXiv: https://arxiv.org/abs/1911.00105 · HTML: https://mlsanity.com/paper/1911.00105/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2019.

## Built on by (most cited first, 18 of 18)

- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [NAAS: Neural Accelerator Architecture Search](https://mlsanity.com/paper/2105.13258/) (2021)
- [The Larger The Fairer? Small Neural Networks Can Achieve Fairness for Edge Devices](https://mlsanity.com/paper/2202.11317/) (2022)
- [Uncertainty Modeling of Emerging Device-based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Search](https://mlsanity.com/paper/2107.06871/) (2021)
- [HAO: Hardware-aware neural Architecture Optimization for Efficient Inference](https://mlsanity.com/paper/2104.12766/) (2021)
- [RankNAS: Efficient Neural Architecture Search by Pairwise Ranking](https://mlsanity.com/paper/2109.07383/) (2021)
- [Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices](https://mlsanity.com/paper/2102.06336/) (2021)
- [AHAR: Adaptive CNN for Energy-efficient Human Activity Recognition in Low-power Edge Devices](https://mlsanity.com/paper/2102.01875/) (2021)
- [RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search](https://mlsanity.com/paper/2109.05691/) (2021)
- [One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2111.01203/) (2021)
- [Trillion Parameter AI Serving Infrastructure for Scientific Discovery: A Survey and Vision](https://mlsanity.com/paper/2402.03480/) (2024)
- [RoHNAS: A Neural Architecture Search Framework with Conjoint Optimization for Adversarial Robustness and Hardware Efficiency of Convolutional and Capsule Networks](https://mlsanity.com/paper/2210.05276/) (2022)
- [RHNAS: Realizable Hardware and Neural Architecture Search](https://mlsanity.com/paper/2106.09180/) (2021)
- [A Semi-Decoupled Approach to Fast and Optimal Hardware-Software Co-Design of Neural Accelerators](https://mlsanity.com/paper/2203.13921/) (2022)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [FAQS: Communication-efficient Federate DNN Architecture and Quantization Co-Search for personalized Hardware-aware Preferences](https://mlsanity.com/paper/2210.08450/) (2022)
- [Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration](https://mlsanity.com/paper/2301.09312/) (2023)
- [A Survey on Multi-Objective Neural Architecture Search](https://mlsanity.com/paper/2307.09099/) (2023)

## Abstract (arXiv)

In the recent past, the success of Neural Architecture Search (NAS) has enabled researchers to broadly explore the design space using learning-based methods. Apart from finding better neural network architectures, the idea of automation has also inspired to improve their implementations on hardware. While some practices of hardware machine-learning automation have achieved remarkable performance, the traditional design concept is still followed: a network architecture is first structured with excellent test accuracy, and then compressed and optimized to fit into a target platform. Such a design flow will easily lead to inferior local-optimal solutions. To address this problem, we propose a new framework to jointly explore the space of neural architecture, hardware implementation, and quantization. Our objective is to find a quantized architecture with the highest accuracy that is implementable on given hardware specifications. We employ FPGAs to implement and test our designs with limited loop-up tables (LUTs) and required throughput. Compared to the separate design/searching methods, our framework has demonstrated much better performance under strict specifications and generated designs of higher accuracy by 18% to 68% in the task of classifying CIFAR10 images. With 30,000 LUTs, a light-weight design is found to achieve 82.98% accuracy and 1293 images/second throughput, compared to which, under the same constraints, the traditional method even fails to find a valid solution.