# SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference

arXiv: https://arxiv.org/abs/2303.08308 · HTML: https://mlsanity.com/paper/2303.08308/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 42 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](https://mlsanity.com/paper/2004.09576/) (2020)
- [BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models](https://mlsanity.com/paper/2003.11142/) (2020)
- [Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.00090/) (2018)
- [APQ: Joint Search for Network Architecture, Pruning and Quantization Policy](https://mlsanity.com/paper/2006.08509/) (2020)
- [Rethinking Differentiable Search for Mixed-Precision Neural Networks](https://mlsanity.com/paper/2004.05795/) (2020)
- [Blockwisely Supervised Neural Architecture Search with Knowledge Distillation](https://mlsanity.com/paper/1911.13053/) (2019)
- [AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling](https://mlsanity.com/paper/2011.09011/) (2020)
- [Searching the Search Space of Vision Transformer](https://mlsanity.com/paper/2111.14725/) (2021)
- [Angle-based Search Space Shrinking for Neural Architecture Search](https://mlsanity.com/paper/2004.13431/) (2020)
- [Distilling Optimal Neural Networks: Rapid Search in Diverse Spaces](https://mlsanity.com/paper/2012.08859/) (2020)
- [ASAP: Architecture Search, Anneal and Prune](https://mlsanity.com/paper/1904.04123/) (2019)
- [Improving One-shot NAS by Suppressing the Posterior Fading](https://mlsanity.com/paper/1910.02543/) (2019)
- [Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search](https://mlsanity.com/paper/2010.04354/) (2020)
- [Evolving Search Space for Neural Architecture Search](https://mlsanity.com/paper/2011.10904/) (2020)
- [BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer](https://mlsanity.com/paper/2105.08952/) (2021)

## Built on by (most cited first, 1 of 1)

- [Deep Learning Inference on Heterogeneous Mobile Processors: Potentials and Pitfalls](https://mlsanity.com/paper/2405.01851/) (2024)

## Abstract (arXiv)

The combination of Neural Architecture Search (NAS) and quantization has proven successful in automatically designing low-FLOPs INT8 quantized neural networks (QNN). However, directly applying NAS to design accurate QNN models that achieve low latency on real-world devices leads to inferior performance. In this work, we find that the poor INT8 latency is due to the quantization-unfriendly issue: the operator and configuration (e.g., channel width) choices in prior art search spaces lead to diverse quantization efficiency and can slow down the INT8 inference speed. To address this challenge, we propose SpaceEvo, an automatic method for designing a dedicated, quantization-friendly search space for each target hardware. The key idea of SpaceEvo is to automatically search hardware-preferred operators and configurations to construct the search space, guided by a metric called Q-T score to quantify how quantization-friendly a candidate search space is. We further train a quantized-for-all supernet over our discovered search space, enabling the searched models to be directly deployed without extra retraining or quantization. Our discovered models establish new SOTA INT8 quantized accuracy under various latency constraints, achieving up to 10.1% accuracy improvement on ImageNet than prior art CNNs under the same latency. Extensive experiments on diverse edge devices demonstrate that SpaceEvo consistently outperforms existing manually-designed search spaces with up to 2.5x faster speed while achieving the same accuracy.