# Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours

arXiv: https://arxiv.org/abs/1904.02877 · HTML: https://mlsanity.com/paper/1904.02877/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2019.

## Built on by (most cited first, 30 of 87)

- [Sustainable AI: Environmental Implications, Challenges and Opportunities](https://mlsanity.com/paper/2111.00364/) (2021)
- [HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark](https://mlsanity.com/paper/2103.10584/) (2021)
- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients](https://mlsanity.com/paper/2301.11300/) (2023)
- [Flextron: Many-in-One Flexible Large Language Model](https://mlsanity.com/paper/2406.10260/) (2024)
- [HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers](https://mlsanity.com/paper/2106.06560/) (2021)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [CompOFA: Compound Once-For-All Networks for Faster Multi-Platform Deployment](https://mlsanity.com/paper/2104.12642/) (2021)
- [Pruning Self-attentions into Convolutional Layers in Single Path](https://mlsanity.com/paper/2111.11802/) (2021)
- [Bit-Mixer: Mixed-precision networks with runtime bit-width selection](https://mlsanity.com/paper/2103.17267/) (2021)
- [Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2307.01998/) (2023)
- [HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search](https://mlsanity.com/paper/2102.11646/) (2021)
- [CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters](https://mlsanity.com/paper/2203.15331/) (2022)
- [Efficient Architecture Search for Diverse Tasks](https://mlsanity.com/paper/2204.07554/) (2022)
- [PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution](https://mlsanity.com/paper/2204.11797/) (2022)
- [The 3D-PC: a benchmark for visual perspective taking in humans and machines](https://mlsanity.com/paper/2406.04138/) (2024)
- [Trilevel Neural Architecture Search for Efficient Single Image Super-Resolution](https://mlsanity.com/paper/2101.06658/) (2021)
- [Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator](https://mlsanity.com/paper/2103.07289/) (2021)
- [Layer Folding: Neural Network Depth Reduction using Activation Linearization](https://mlsanity.com/paper/2106.09309/) (2021)
- [NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimization](https://mlsanity.com/paper/2104.00031/) (2021)
- [Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators](https://mlsanity.com/paper/2106.06575/) (2021)
- [DropNAS: Grouped Operation Dropout for Differentiable Architecture Search](https://mlsanity.com/paper/2201.11679/) (2022)
- [FLASH: Fast Neural Architecture Search with Hardware Optimization](https://mlsanity.com/paper/2108.00568/) (2021)
- [Learning Versatile Neural Architectures by Propagating Network Codes](https://mlsanity.com/paper/2103.13253/) (2021)
- [EEEA-Net: An Early Exit Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2108.06156/) (2021)
- [Searching for Efficient Multi-Stage Vision Transformers](https://mlsanity.com/paper/2109.00642/) (2021)
- [FL-NAS: Towards Fairness of NAS for Resource Constrained Devices via Large Language Models](https://mlsanity.com/paper/2402.06696/) (2024)
- [One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2111.01203/) (2021)
- [EAutoDet: Efficient Architecture Search for Object Detection](https://mlsanity.com/paper/2203.10747/) (2022)

## Abstract (arXiv)

Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the runtime constraint of a mobile device? Neural architecture search (NAS) has revolutionized the design of hardware-efficient ConvNets by automating this process. However, the NAS problem remains challenging due to the combinatorially large design space, causing a significant searching time (at least 200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a novel differentiable NAS method for designing hardware-efficient ConvNets in less than 4 hours. Our contributions are as follows: 1. Single-path search space: Compared to previous differentiable NAS methods, Single-Path NAS uses one single-path over-parameterized ConvNet to encode all architectural decisions with shared convolutional kernel parameters, hence drastically decreasing the number of trainable parameters and the search cost down to few epochs. 2. Hardware-efficient ImageNet classification: Single-Path NAS achieves 74.96% top-1 accuracy on ImageNet with 79ms latency on a Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with similar constraints (<80ms). 3. NAS efficiency: Single-Path NAS search cost is only 8 epochs (30 TPU-hours), which is up to 5,000x faster compared to prior work. 4. Reproducibility: Unlike all recent mobile-efficient NAS methods which only release pretrained models, we open-source our entire codebase at: https://github.com/dstamoulis/single-path-nas.