mlsanityData to 7 Oct 2026

Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization

arXiv 1902.01917 ↗First posted 5 Feb 2019cs.LG stat.MLcited by 45

Not on the map. Why?

Abstract (arXiv)

Quantization of neural networks has become common practice, driven by the need for efficient implementations of deep neural networks on embedded devices. In this paper, we exploit an oft-overlooked degree of freedom in most networks - for a given layer, individual output channels can be scaled by any factor provided that the corresponding weights of the next layer are inversely scaled. Therefore, a given network has many factorizations which change the weights of the network without changing its function. We present a conceptually simple and easy to implement method that uses this property and show that proper factorizations significantly decrease the degradation caused by quantization. We show improvement on a wide variety of networks and achieve state-of-the-art degradation results for MobileNets. While our focus is on quantization, this type of factorization is applicable to other domains such as network-pruning, neural nets regularization and network interpretability.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 8; 2023: 7; 2024: 8; 2025: 5; 2026: 4.

45 papers.

Sort:

By area:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  2. 2021A White Paper on Neural Network Quantizationcited by 453
  3. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
  4. 2021Zero-shot Adversarial Quantizationcited by 44
  5. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  6. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  7. 2021Ps and Qs: Quantization-aware pruning for efficient low latency neural network inferencecited by 31
  8. 2025Symmetry in Neural Network Parameter Spacescited by 31
  9. 2021An Embedding of ReLU Networks and an Analysis of their Identifiabilitycited by 20
  10. 2021RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networkscited by 19
  11. 2021RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Mergingcited by 19
  12. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
33 more
  1. 2021Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamicscited by 12
  2. 2022SPIQ: Data-Free Per-Channel Static Input Quantizationcited by 12
  3. 2021HPTQ: Hardware-Friendly Post Training Quantizationcited by 9
  4. 2021Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scalecited by 7
  5. 2021A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statisticscited by 6
  6. 2022Quantune: Post-training Quantization of Convolutional Neural Networks using Extreme Gradient Boosting for Fast Deploymentcited by 6
  7. 2022REx: Data-Free Residual Quantization Error Expansioncited by 6
  8. 2023Q-HyViT: Post-Training Quantization of Hybrid Vision Transformers with Bridge Block Reconstruction for IoT Systemscited by 6
  9. 2023Quantization Aware Factorization for Deep Neural Network Compressioncited by 6
  10. 2024Quantized Prompt for Efficient Generalization of Vision-Language Modelscited by 5
  11. 2026SliderQuant: Accurate Post-Training Quantization for LLMscited by 5
  12. 2022To Fold or Not to Fold: a Necessary and Sufficient Condition on Batch-Normalization Layers Foldingcited by 4
  13. 2024Accumulator-Aware Post-Training Quantization for Large Language Modelscited by 4
  14. 2022MinUn: Accurate ML Inference on Microcontrollerscited by 3
  15. 2022QFT: Post-training quantization via fast joint finetuning of all degrees of freedomcited by 3
  16. 2023Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAscited by 3
  17. 2024On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performancecited by 3
  18. 2024BBS: Bi-directional Bit-level Sparsity for Deep Learning Accelerationcited by 3
  19. 2021Automated Backend-Aware Post-Training Quantizationcited by 2
  20. 2022FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representationcited by 2
  21. 2023ResQ: Residual Quantization for Video Perceptioncited by 2
  22. 2025QuantuneV2: Compiler-Based Local Metric-Driven Mixed Precision Quantization for Practical Embedded AI Applicationscited by 2
  23. 2024Quantization of Large Language Models with an Overdetermined Basiscited by 1
  24. 2024On the Impact of White-box Deployment Strategies for Edge AI on Latency and Model Performancecited by 1
  25. 2025Sensitivity-Aware Post-Training Quantization for Deep Neural Networkscited by 1
  26. 2025SkipCat: Rank-Maximized Low-Rank Compression of Large Language Models via Shared Projection and Block Skippingcited by 1
  27. 2026Preserve-Then-Quantize: Balancing Rank Budgets for Quantization Error Reconstruction in LLMscited by 1
  28. 2026Contraction-Gauge Preconditioning for Quantized Matrix Multiplicationcited by 1
  29. 2022UWC: Unit-wise Calibration Towards Rapid Network Compressioncited by 0
  30. 2023Designing strong baselines for ternary neural network quantization through support and mass equalizationcited by 0
  31. 2023PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansionscited by 0
  32. 2024DNN Memory Footprint Reduction via Post-Training Intra-Layer Multi-Precision Quantizationcited by 0
  33. 2026Empowering On-Device Model Adaptation with an Edge AI Inference Acceleratorcited by 0