# Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance

arXiv: https://arxiv.org/abs/2203.08368 · HTML: https://mlsanity.com/paper/2203.08368/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 30 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks](https://mlsanity.com/paper/1911.03852/) (2019)
- [Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights](https://mlsanity.com/paper/1702.03044/) (2017)
- [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](https://mlsanity.com/paper/2004.09576/) (2020)
- [Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss](https://mlsanity.com/paper/1808.05779/) (2018)
- [FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search](https://mlsanity.com/paper/1907.01845/) (2019)
- [Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.00090/) (2018)
- [Deep Learning with Low Precision by Half-wave Gaussian Quantization](https://mlsanity.com/paper/1702.00953/) (2017)
- [Rethinking Differentiable Search for Mixed-Precision Neural Networks](https://mlsanity.com/paper/2004.05795/) (2020)
- [AutoQ: Automated Kernel-Wise Neural Network Quantization](https://mlsanity.com/paper/1902.05690/) (2019)
- [Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization](https://mlsanity.com/paper/2110.06554/) (2021)
- [PROFIT: A Novel Training Method for sub-4-bit MobileNet Models](https://mlsanity.com/paper/2008.04693/) (2020)
- [Differentiable Model Compression via Pseudo Quantization Noise](https://mlsanity.com/paper/2104.09987/) (2021)
- [HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNs](https://mlsanity.com/paper/2007.09952/) (2020)
- [Search What You Want: Barrier Panelty NAS for Mixed Precision Quantization](https://mlsanity.com/paper/2007.10026/) (2020)
- [NICE: Noise Injection and Clamping Estimation for Neural Network Quantization](https://mlsanity.com/paper/1810.00162/) (2018)
- [BN-NAS: Neural Architecture Search with Batch Normalization](https://mlsanity.com/paper/2108.07375/) (2021)

## Built on by (most cited first, 30 of 54)

- [Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity](https://mlsanity.com/paper/2310.05175/) (2023)
- [Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers](https://mlsanity.com/paper/2406.17343/) (2024)
- [LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning](https://mlsanity.com/paper/2311.12023/) (2023)
- [NetLLM: Adapting Large Language Models for Networking](https://mlsanity.com/paper/2402.02338/) (2024)
- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration](https://mlsanity.com/paper/2506.12723/) (2025)
- [MesonGS: Post-training Compression of 3D Gaussians via Efficient Attribute Transformation](https://mlsanity.com/paper/2409.09756/) (2024)
- [AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models](https://mlsanity.com/paper/2410.10912/) (2024)
- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [Data Quality-aware Mixed-precision Quantization via Hybrid Reinforcement Learning](https://mlsanity.com/paper/2302.04453/) (2023)
- [LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inference](https://mlsanity.com/paper/2406.19657/) (2024)
- [Retraining-free Model Quantization via One-Shot Weight-Coupling Learning](https://mlsanity.com/paper/2401.01543/) (2024)
- [FIT: A Metric for Model Sensitivity](https://mlsanity.com/paper/2210.08502/) (2022)
- [Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices](https://mlsanity.com/paper/2311.18129/) (2023)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [TMPQ-DM: Joint Timestep Reduction and Quantization Precision Selection for Efficient Diffusion Models](https://mlsanity.com/paper/2404.09532/) (2024)
- [PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference](https://mlsanity.com/paper/2407.05010/) (2024)
- [AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution](https://mlsanity.com/paper/2404.03296/) (2024)
- [Investigating the Impact of Quantization on Adversarial Robustness](https://mlsanity.com/paper/2404.05639/) (2024)
- [Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity](https://mlsanity.com/paper/2501.06357/) (2025)
- [Win Fast or Lose Slow: Balancing Speed and Accuracy in Latency-Sensitive Decisions of LLMs](https://mlsanity.com/paper/2505.19481/) (2025)
- [Mixed-Precision Quantization for Deep Vision Models with Integer Quadratic Programming](https://mlsanity.com/paper/2307.05657/) (2023)
- [TransAxx: Efficient Transformers with Approximate Computing](https://mlsanity.com/paper/2402.07545/) (2024)
- [Content-Aware Radiance Fields: Aligning Model Complexity with Scene Intricacy Through Learned Bitwidth Quantization](https://mlsanity.com/paper/2410.19483/) (2024)
- [SEAM: Searching Transferable Mixed-Precision Quantization Policy through Large Margin Regularization](https://mlsanity.com/paper/2302.06845/) (2023)
- [PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models](https://mlsanity.com/paper/2306.00014/) (2023)
- [Data-Aware Gradient Compression for FL in Communication-Constrained Mobile Computing](https://mlsanity.com/paper/2311.07324/) (2023)
- [Towards Optimal Compression: Joint Pruning and Quantization](https://mlsanity.com/paper/2302.07612/) (2023)
- [Mix-QSAM: Mixed-Precision Quantization of the Segment Anything Model](https://mlsanity.com/paper/2505.04861/) (2025)
- [InfoQ: Mixed-Precision Quantization via Global Information Flow](https://mlsanity.com/paper/2508.04753/) (2025)

## Abstract (arXiv)

The exponentially large discrete search space in mixed-precision quantization (MPQ) makes it hard to determine the optimal bit-width for each layer. Previous works usually resort to iterative search methods on the training set, which consume hundreds or even thousands of GPU-hours. In this study, we reveal that some unique learnable parameters in quantization, namely the scale factors in the quantizer, can serve as importance indicators of a layer, reflecting the contribution of that layer to the final accuracy at certain bit-widths. These importance indicators naturally perceive the numerical transformation during quantization-aware training, which can precisely provide quantization sensitivity metrics of layers. However, a deep network always contains hundreds of such indicators, and training them one by one would lead to an excessive time cost. To overcome this issue, we propose a joint training scheme that can obtain all indicators at once. It considerably speeds up the indicators training process by parallelizing the original sequential training processes. With these learned importance indicators, we formulate the MPQ search problem as a one-time integer linear programming (ILP) problem. That avoids the iterative search and significantly reduces search time without limiting the bit-width search space. For example, MPQ search on ResNet18 with our indicators takes only 0.06 s, which improves time efficiency exponentially compared to iterative search methods. Also, extensive experiments show our approach can achieve SOTA accuracy on ImageNet for far-ranging models with various constraints (e.g., BitOps, compress rate). Code is available on https://github.com/1hunters/LIMPQ.