# Finding Non-Uniform Quantization Schemes using Multi-Task Gaussian Processes

arXiv: https://arxiv.org/abs/2007.07743 · HTML: https://mlsanity.com/paper/2007.07743/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2020.

## Built on by (most cited first, 5 of 5)

- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification](https://mlsanity.com/paper/2212.02770/) (2022)
- [Single-DARTS: Towards Stable Architecture Search](https://mlsanity.com/paper/2108.08128/) (2021)
- [Learning Robust and Lightweight Model through Separable Structured Transformations](https://mlsanity.com/paper/2112.13551/) (2021)
- [Approximating Continuous Convolutions for Deep Network Compression](https://mlsanity.com/paper/2210.08951/) (2022)

## Abstract (arXiv)

We propose a novel method for neural network quantization that casts the neural architecture search problem as one of hyperparameter search to find non-uniform bit distributions throughout the layers of a CNN. We perform the search assuming a Multi-Task Gaussian Processes prior, which splits the problem to multiple tasks, each corresponding to different number of training epochs, and explore the space by sampling those configurations that yield maximum information. We then show that with significantly lower precision in the last layers we achieve a minimal loss of accuracy with appreciable memory savings. We test our findings on the CIFAR10 and ImageNet datasets using the VGG, ResNet and GoogLeNet architectures.