# APQ: Joint Search for Network Architecture, Pruning and Quantization Policy

arXiv: https://arxiv.org/abs/2006.08509 · HTML: https://mlsanity.com/paper/2006.08509/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2020.

## Built on by (most cited first, 30 of 99)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits](https://mlsanity.com/paper/2107.10845/) (2021)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [A Survey on Transformer Compression](https://mlsanity.com/paper/2402.05964/) (2024)
- [EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction](https://mlsanity.com/paper/2205.14756/) (2022)
- [Structural Pruning via Latency-Saliency Knapsack](https://mlsanity.com/paper/2210.06659/) (2022)
- [When to Prune? A Policy towards Early Structural Pruning](https://mlsanity.com/paper/2110.12007/) (2021)
- [Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search](https://mlsanity.com/paper/2108.08910/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning](https://mlsanity.com/paper/2102.03214/) (2021)
- [GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization](https://mlsanity.com/paper/2109.02220/) (2021)
- [Sharpness-aware Quantization for Deep Neural Networks](https://mlsanity.com/paper/2111.12273/) (2021)
- [QOC: Quantum On-Chip Training with Parameter Shift and Gradient Pruning](https://mlsanity.com/paper/2202.13239/) (2022)
- [Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search](https://mlsanity.com/paper/2206.09811/) (2022)
- [EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization](https://mlsanity.com/paper/2307.10554/) (2023)
- [BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer](https://mlsanity.com/paper/2105.08952/) (2021)
- [OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization](https://mlsanity.com/paper/2205.11141/) (2022)
- [Exploring Incompatible Knowledge Transfer in Few-shot Image Generation](https://mlsanity.com/paper/2304.07574/) (2023)
- [Effective Interplay between Sparsity and Quantization: From Theory to Practice](https://mlsanity.com/paper/2405.20935/) (2024)
- [PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution](https://mlsanity.com/paper/2204.11797/) (2022)
- [Efficient Quantized Sparse Matrix Operations on Tensor Cores](https://mlsanity.com/paper/2209.06979/) (2022)
- [Model-Based Control with Sparse Neural Dynamics](https://mlsanity.com/paper/2312.12791/) (2023)
- [Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression](https://mlsanity.com/paper/2502.16638/) (2025)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators](https://mlsanity.com/paper/2106.06575/) (2021)
- [Generalizable Mixed-Precision Quantization via Attribution Rank Preservation](https://mlsanity.com/paper/2108.02720/) (2021)
- [T-AutoML: Automated Machine Learning for Lesion Segmentation using Transformers in 3D Medical Imaging](https://mlsanity.com/paper/2111.07535/) (2021)
- [SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning](https://mlsanity.com/paper/2111.14345/) (2021)

## Abstract (arXiv)

We present APQ for efficient deep learning inference on resource-constrained hardware. Unlike previous methods that separately search the neural architecture, pruning policy, and quantization policy, we optimize them in a joint manner. To deal with the larger design space it brings, a promising approach is to train a quantization-aware accuracy predictor to quickly get the accuracy of the quantized model and feed it to the search engine to select the best fit. However, training this quantization-aware accuracy predictor requires collecting a large number of quantized <model, accuracy> pairs, which involves quantization-aware finetuning and thus is highly time-consuming. To tackle this challenge, we propose to transfer the knowledge from a full-precision (i.e., fp32) accuracy predictor to the quantization-aware (i.e., int8) accuracy predictor, which greatly improves the sample efficiency. Besides, collecting the dataset for the fp32 accuracy predictor only requires to evaluate neural networks without any training cost by sampling from a pretrained once-for-all network, which is highly efficient. Extensive experiments on ImageNet demonstrate the benefits of our joint optimization approach. With the same accuracy, APQ reduces the latency/energy by 2x/1.3x over MobileNetV2+HAQ. Compared to the separate optimization approach (ProxylessNAS+AMC+HAQ), APQ achieves 2.3% higher ImageNet accuracy while reducing orders of magnitude GPU hours and CO2 emission, pushing the frontier for green AI that is environmental-friendly. The code and video are publicly available.