mlsanityData to 14 Oct 2026

StatQAT: Statistical Quantizer Optimization for Deep Networks

arXiv 2605.17745 ↗First posted 18 May 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Quantization is essential for reducing the computational cost and memory usage of deep neural networks, enabling efficient inference on low-precision hardware. Despite the growing adoption of uniform and floating-point quantization schemes, selecting optimal quantization parameters remains a key challenge, particularly for diverse data distributions encountered during training and inference. This work presents a novel statistical error analysis framework for uniform and floating-point quantization, providing theoretical insight into error behavior across quantization configurations. Building on this analysis, we propose iterative quantizers designed for arbitrary data distributions and analytic quantizers tailored for Gaussian-like weight distributions. These methods enable efficient, low-error quantization suitable for both activations and weights. We incorporate our quantizers into quantization-aware training and evaluate them across integer and floating-point formats. Experiments demonstrate improved accuracy and stability, highlighting the effectiveness of our approach for training low-precision neural networks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

18 of 28 references.

References · 18

  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  2. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  3. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  5. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  6. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  7. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  8. 2019Learned Step Size Quantizationcited by 628
  9. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  10. 2015Weight Uncertainty in Neural Networkscited by 554
  11. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  12. 2022FP8 Formats for Deep Learningcited by 196
6 more
  1. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  2. 2022FP8 Quantization: The Power of the Exponentcited by 88
  3. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 69
  4. 2022Optimal Clipping and Magnitude-aware Differentiation for Improved Quantization-aware Trainingcited by 26
  5. 2018Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)cited by 19
  6. 2019Mixed Precision DNNs: All you need is a good parametrizationcited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021A White Paper on Neural Network Quantization
  2. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
  3. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
  4. 2025Dual Precision Quantization for Efficient and Accurate Deep Neural Networks Inference
  5. 2023Gradient-Based Post-Training Quantization: Challenging the Status Quo
  6. 2023Designing strong baselines for ternary neural network quantization through support and mass equalization
  7. 2025Gaussian Weight Sampling for Scalable, Efficient and Stable Pseudo-Quantization Training
  8. 2021Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms