# MixQuant: Mixed Precision Quantization with a Bit-width Optimization Search

arXiv: https://arxiv.org/abs/2309.17341 · HTML: https://mlsanity.com/paper/2309.17341/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 28 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Builds on (most cited first, 22 of 22)

- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [Low-bit Quantization of Neural Networks for Efficient Inference](https://mlsanity.com/paper/1902.06822/) (2019)
- [Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks](https://mlsanity.com/paper/1909.13144/) (2019)
- [Fixed Point Quantization of Deep Convolutional Networks](https://mlsanity.com/paper/1511.06393/) (2015)
- [Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.00090/) (2018)
- [Loss Aware Post-training Quantization](https://mlsanity.com/paper/1911.07190/) (2019)
- [Degree-Quant: Quantization-Aware Training for Graph Neural Networks](https://mlsanity.com/paper/2008.05000/) (2020)
- [Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming](https://mlsanity.com/paper/2006.10518/) (2020)
- [Differentiable Product Quantization for End-to-End Embedding Compression](https://mlsanity.com/paper/1908.09756/) (2019)
- [Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision](https://mlsanity.com/paper/2002.09049/) (2020)
- [DeepStability: A Study of Unstable Numerical Methods and Their Solutions in Deep Learning](https://mlsanity.com/paper/2202.03493/) (2022)
- [Training Quantized Neural Networks to Global Optimality via Semidefinite Programming](https://mlsanity.com/paper/2105.01420/) (2021)
- [Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks](https://mlsanity.com/paper/1906.06033/) (2019)

## Built on by (most cited first, 4 of 4)

- [Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivity](https://mlsanity.com/paper/2603.17354/) (2026)
- [AlignedKV: Reducing Memory Access of KV-Cache with Precision-Aligned Quantization](https://mlsanity.com/paper/2409.16546/) (2024)
- [M2-ViT: Accelerating Hybrid Vision Transformers with Two-Level Mixed Quantization](https://mlsanity.com/paper/2410.09113/) (2024)
- [Explore as a Storm, Exploit as a Raindrop: On the Benefit of Fine-Tuning Kernel Schedulers with Coordinate Descent](https://mlsanity.com/paper/2406.20037/) (2024)

## Abstract (arXiv)

Quantization is a technique for creating efficient Deep Neural Networks (DNNs), which involves performing computations and storing tensors at lower bit-widths than f32 floating point precision. Quantization reduces model size and inference latency, and therefore allows for DNNs to be deployed on platforms with constrained computational resources and real-time systems. However, quantization can lead to numerical instability caused by roundoff error which leads to inaccurate computations and therefore, a decrease in quantized model accuracy. Similarly to prior works, which have shown that both biases and activations are more sensitive to quantization and are best kept in full precision or quantized with higher bit-widths, we show that some weights are more sensitive than others which should be reflected on their quantization bit-width. To that end we propose MixQuant, a search algorithm that finds the optimal custom quantization bit-width for each layer weight based on roundoff error and can be combined with any quantization method as a form of pre-processing optimization. We show that combining MixQuant with BRECQ, a state-of-the-art quantization method, yields better quantized model accuracy than BRECQ alone. Additionally, we combine MixQuant with vanilla asymmetric quantization to show that MixQuant has the potential to optimize the performance of any quantization technique.