# Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization

arXiv: https://arxiv.org/abs/1902.01917 · HTML: https://mlsanity.com/paper/1902.01917/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2019.

## Built on by (most cited first, 30 of 45)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing](https://mlsanity.com/paper/2306.12929/) (2023)
- [Zero-shot Adversarial Quantization](https://mlsanity.com/paper/2103.15263/) (2021)
- [Low-Rank Quantization-Aware Training for LLMs](https://mlsanity.com/paper/2406.06385/) (2024)
- [Symmetry in Neural Network Parameter Spaces](https://mlsanity.com/paper/2506.13018/) (2025)
- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [Ps and Qs: Quantization-aware pruning for efficient low latency neural network inference](https://mlsanity.com/paper/2102.11289/) (2021)
- [An Embedding of ReLU Networks and an Analysis of their Identifiability](https://mlsanity.com/paper/2107.09370/) (2021)
- [RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networks](https://mlsanity.com/paper/2105.14797/) (2021)
- [RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging](https://mlsanity.com/paper/2110.01397/) (2021)
- [FPTQuant: Function-Preserving Transforms for LLM Quantization](https://mlsanity.com/paper/2506.04985/) (2025)
- [Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamics](https://mlsanity.com/paper/2104.11849/) (2021)
- [SPIQ: Data-Free Per-Channel Static Input Quantization](https://mlsanity.com/paper/2203.14642/) (2022)
- [HPTQ: Hardware-Friendly Post Training Quantization](https://mlsanity.com/paper/2109.09113/) (2021)
- [Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale](https://mlsanity.com/paper/2105.12676/) (2021)
- [A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics](https://mlsanity.com/paper/2112.02834/) (2021)
- [Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for Fast Deployment](https://mlsanity.com/paper/2202.05048/) (2022)
- [REx: Data-Free Residual Quantization Error Expansion](https://mlsanity.com/paper/2203.14645/) (2022)
- [Q-HyViT: Post-Training Quantization of Hybrid Vision Transformers with Bridge Block Reconstruction for IoT Systems](https://mlsanity.com/paper/2303.12557/) (2023)
- [Quantization Aware Factorization for Deep Neural Network Compression](https://mlsanity.com/paper/2308.04595/) (2023)
- [Quantized Prompt for Efficient Generalization of Vision-Language Models](https://mlsanity.com/paper/2407.10704/) (2024)
- [Accumulator-Aware Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2409.17092/) (2024)
- [SliderQuant: Accurate Post-Training Quantization for LLMs](https://mlsanity.com/paper/2603.25284/) (2026)
- [To Fold or Not to Fold: a Necessary and Sufficient Condition on Batch-Normalization Layers Folding](https://mlsanity.com/paper/2203.14646/) (2022)
- [Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAs](https://mlsanity.com/paper/2311.12359/) (2023)
- [MinUn: Accurate ML Inference on Microcontrollers](https://mlsanity.com/paper/2210.16556/) (2022)
- [QFT: Post-training quantization via fast joint finetuning of all degrees of freedom](https://mlsanity.com/paper/2212.02634/) (2022)
- [On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance](https://mlsanity.com/paper/2403.17154/) (2024)
- [BBS: Bi-directional Bit-level Sparsity for Deep Learning Acceleration](https://mlsanity.com/paper/2409.05227/) (2024)

## Abstract (arXiv)

Quantization of neural networks has become common practice, driven by the need for efficient implementations of deep neural networks on embedded devices. In this paper, we exploit an oft-overlooked degree of freedom in most networks - for a given layer, individual output channels can be scaled by any factor provided that the corresponding weights of the next layer are inversely scaled. Therefore, a given network has many factorizations which change the weights of the network without changing its function. We present a conceptually simple and easy to implement method that uses this property and show that proper factorizations significantly decrease the degradation caused by quantization. We show improvement on a wide variety of networks and achieve state-of-the-art degradation results for MobileNets. While our focus is on quantization, this type of factorization is applicable to other domains such as network-pruning, neural nets regularization and network interpretability.