# Weight Normalization based Quantization for Deep Neural Network Compression

arXiv: https://arxiv.org/abs/1907.00593 · HTML: https://mlsanity.com/paper/1907.00593/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2019.

## Built on by (most cited first, 5 of 5)

- [A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidance](https://mlsanity.com/paper/2308.13504/) (2023)
- [Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attacks](https://mlsanity.com/paper/2107.06304/) (2021)
- [Quantized Convolutional Neural Networks Through the Lens of Partial Differential Equations](https://mlsanity.com/paper/2109.00095/) (2021)
- [Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance](https://mlsanity.com/paper/2301.13376/) (2023)
- [GradFreeBits: Gradient Free Bit Allocation for Dynamic Low Precision Neural Networks](https://mlsanity.com/paper/2102.09298/) (2021)

## Abstract (arXiv)

With the development of deep neural networks, the size of network models becomes larger and larger. Model compression has become an urgent need for deploying these network models to mobile or embedded devices. Model quantization is a representative model compression technique. Although a lot of quantization methods have been proposed, many of them suffer from a high quantization error caused by a long-tail distribution of network weights. In this paper, we propose a novel quantization method, called weight normalization based quantization (WNQ), for model compression. WNQ adopts weight normalization to avoid the long-tail distribution of network weights and subsequently reduces the quantization error. Experiments on CIFAR-100 and ImageNet show that WNQ can outperform other baselines to achieve state-of-the-art performance.