# Joint Neural Architecture Search and Quantization

arXiv: https://arxiv.org/abs/1811.09426 · HTML: https://mlsanity.com/paper/1811.09426/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2018.

## Built on by (most cited first, 13 of 13)

- [BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer](https://mlsanity.com/paper/2105.08952/) (2021)
- [Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators](https://mlsanity.com/paper/2106.06575/) (2021)
- [Mixed-Precision Neural Networks: A Survey](https://mlsanity.com/paper/2208.06064/) (2022)
- [BOMP-NAS: Bayesian Optimization Mixed Precision NAS](https://mlsanity.com/paper/2301.11810/) (2023)
- [BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements](https://mlsanity.com/paper/2110.08562/) (2021)
- [Efficient and Effective Methods for Mixed Precision Neural Network Quantization for Faster, Energy-efficient Inference](https://mlsanity.com/paper/2301.13330/) (2023)
- [InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks](https://mlsanity.com/paper/2104.10853/) (2021)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [Combining Neural Architecture Search and Automatic Code Optimization: A Survey](https://mlsanity.com/paper/2408.04116/) (2024)
- [SpeedLimit: Neural Architecture Search for Quantized Transformer Models](https://mlsanity.com/paper/2209.12127/) (2022)
- [FAQS: Communication-efficient Federate DNN Architecture and Quantization Co-Search for personalized Hardware-aware Preferences](https://mlsanity.com/paper/2210.08450/) (2022)
- [HQNAS: Auto CNN deployment framework for joint quantization and architecture search](https://mlsanity.com/paper/2210.08485/) (2022)
- [A Survey on Multi-Objective Neural Architecture Search](https://mlsanity.com/paper/2307.09099/) (2023)

## Abstract (arXiv)

Designing neural architectures is a fundamental step in deep learning applications. As a partner technique, model compression on neural networks has been widely investigated to gear the needs that the deep learning algorithms could be run with the limited computation resources on mobile devices. Currently, both the tasks of architecture design and model compression require expertise tricks and tedious trials. In this paper, we integrate these two tasks into one unified framework, which enables the joint architecture search with quantization (compression) policies for neural networks. This method is named JASQ. Here our goal is to automatically find a compact neural network model with high performance that is suitable for mobile devices. Technically, a multi-objective evolutionary search algorithm is introduced to search the models under the balance between model size and performance accuracy. In experiments, we find that our approach outperforms the methods that search only for architectures or only for quantization policies. 1) Specifically, given existing networks, our approach can provide them with learning-based quantization policies, and outperforms their 2 bits, 4 bits, 8 bits, and 16 bits counterparts. It can yield higher accuracies than the float models, for example, over 1.02% higher accuracy on MobileNet-v1. 2) What is more, under the balance between model size and performance accuracy, two models are obtained with joint search of architectures and quantization policies: a high-accuracy model and a small model, JASQNet and JASQNet-Small that achieves 2.97% error rate with 0.9 MB on CIFAR-10.