# Bayesian Bits: Unifying Quantization and Pruning

arXiv: https://arxiv.org/abs/2005.07093 · HTML: https://mlsanity.com/paper/2005.07093/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2020.

## Built on by (most cited first, 30 of 94)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [Understanding and Overcoming the Challenges of Efficient Transformer Quantization](https://mlsanity.com/paper/2109.12948/) (2021)
- [Learned Token Pruning for Transformers](https://mlsanity.com/paper/2107.00910/) (2021)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Only Train Once: A One-Shot Neural Network Training And Pruning Framework](https://mlsanity.com/paper/2107.07467/) (2021)
- [Implicit Neural Video Compression](https://mlsanity.com/paper/2112.11312/) (2021)
- [Pruning vs Quantization: Which is Better?](https://mlsanity.com/paper/2307.02973/) (2023)
- [PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers](https://mlsanity.com/paper/2209.05687/) (2022)
- [MQBench: Towards Reproducible and Deployable Model Quantization Benchmark](https://mlsanity.com/paper/2111.03759/) (2021)
- [OTOV2: Automatic, Generic, User-Friendly](https://mlsanity.com/paper/2303.06862/) (2023)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [Neural Network Quantization with AI Model Efficiency Toolkit (AIMET)](https://mlsanity.com/paper/2201.08442/) (2022)
- [A Practical Mixed Precision Algorithm for Post-Training Quantization](https://mlsanity.com/paper/2302.05397/) (2023)
- [QVGen: Pushing the Limit of Quantized Video Generative Models](https://mlsanity.com/paper/2505.11497/) (2025)
- [Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks](https://mlsanity.com/paper/2203.03844/) (2022)
- [BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer](https://mlsanity.com/paper/2105.08952/) (2021)
- [Integer-only Zero-shot Quantization for Efficient Speech Recognition](https://mlsanity.com/paper/2103.16827/) (2021)
- [Efficient Quantized Sparse Matrix Operations on Tensor Cores](https://mlsanity.com/paper/2209.06979/) (2022)
- [Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression](https://mlsanity.com/paper/2502.16638/) (2025)
- [Retraining-free Model Quantization via One-Shot Weight-Coupling Learning](https://mlsanity.com/paper/2401.01543/) (2024)
- [Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices](https://mlsanity.com/paper/2206.10844/) (2022)
- [Generalizable Mixed-Precision Quantization via Attribution Rank Preservation](https://mlsanity.com/paper/2108.02720/) (2021)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization](https://mlsanity.com/paper/2312.15322/) (2023)
- [Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks](https://mlsanity.com/paper/2112.15139/) (2021)
- [Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2311.11883/) (2023)
- [On the Effects of Quantisation on Model Uncertainty in Bayesian Neural Networks](https://mlsanity.com/paper/2102.11062/) (2021)
- [Dynamic ConvNets on Tiny Devices via Nested Sparsity](https://mlsanity.com/paper/2203.03324/) (2022)

## Abstract (arXiv)

We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization. Bayesian Bits employs a novel decomposition of the quantization operation, which sequentially considers doubling the bit width. At each new bit width, the residual error between the full precision value and the previously rounded value is quantized. We then decide whether or not to add this quantized residual error for a higher effective bit width and lower quantization noise. By starting with a power-of-two bit width, this decomposition will always produce hardware-friendly configurations, and through an additional 0-bit option, serves as a unified view of pruning and quantization. Bayesian Bits then introduces learnable stochastic gates, which collectively control the bit width of the given tensor. As a result, we can obtain low bit solutions by performing approximate inference over the gates, with prior distributions that encourage most of them to be switched off. We experimentally validate our proposed method on several benchmark datasets and show that we can learn pruned, mixed precision networks that provide a better trade-off between accuracy and efficiency than their static bit width equivalents.