mlsanityData to 7 Oct 2026

A White Paper on Neural Network Quantization

arXiv 2106.08295 ↗First posted 15 Jun 2021cs.LG cs.AI cs.CVcited by 453

Not on the map. Why?

Cited by 33 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While neural networks have advanced the frontiers in many applications, they often come at a high computational cost. Reducing the power and latency of neural network inference is key if we want to integrate modern networks into edge devices with strict power and compute requirements. Neural network quantization is one of the most effective ways of achieving these savings but the additional noise it induces can lead to accuracy degradation. In this white paper, we introduce state-of-the-art algorithms for mitigating the impact of quantization noise on the network's performance while maintaining low-bit weights and activations. We start with a hardware motivated introduction to quantization and then consider two main classes of algorithms: Post-Training Quantization (PTQ) and Quantization-Aware-Training (QAT). PTQ requires no re-training or labelled data and is thus a lightweight push-button approach to quantization. In most cases, PTQ is sufficient for achieving 8-bit quantization with close to floating-point accuracy. QAT requires fine-tuning and access to labeled training data but enables lower bit quantization with competitive results. For both solutions, we provide tested pipelines based on existing literature and extensive experimentation that lead to state-of-the-art performance for common deep learning models and tasks.

Key facts. Cited by 453 papers in the map (data through 7 Oct 2026). Builds on 23 of its 39 references that are in the map. First posted on arXiv 15 Jun 2021.

Builds on

23 of 39 references.

References · 23

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  5. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  8. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  9. 2019Learned Step Size Quantizationcited by 604
  10. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  11. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  12. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
11 more
  1. 2015Deep Learning with Limited Numerical Precisioncited by 368
  2. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  3. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  4. 2020Bayesian Bits: Unifying Quantization and Pruningcited by 93
  5. 2019And the Bit Goes Down: Revisiting the Quantization of Neural Networkscited by 78
  6. 2019Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorizationcited by 45
  7. 2020Distilling Optimal Neural Networks: Rapid Search in Diverse Spacescited by 32
  8. 2019Fighting Quantization Bias With Biascited by 30
  9. 2018A Quantization-Friendly Separable Convolution for MobileNetscited by 22
  10. 2020One Weight Bitwidth to Rule Them Allcited by 13
  11. 2019DSConv: Efficient Convolution Operatorcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 34; 2023: 69; 2024: 120; 2025: 125; 2026: 99.

453 papers.

Sort:

By area:

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  3. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  4. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  5. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 266
  6. 2022Post-training Quantization on Diffusion Modelscited by 262
  7. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
  8. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  9. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 116
  10. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 110
  11. 2022FP8 Quantization: The Power of the Exponentcited by 87
  12. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 87
38 more
  1. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  2. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
  3. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 77
  4. 2024Evaluating Quantized Large Language Modelscited by 75
  5. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  6. 2023Stateful Large Language Model Serving with Pensievecited by 59
  7. 2024Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMscited by 54
  8. 2021Implicit Neural Video Compressioncited by 52
  9. 2023Intriguing Properties of Quantization at Scalecited by 48
  10. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 46
  11. 2023Pruning vs Quantization: Which is Better?cited by 45
  12. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 44
  13. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 42
  14. 2023Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based acceleratorscited by 42
  15. 2023QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Modelscited by 42
  16. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantizationcited by 42
  17. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  18. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
  19. 2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Modelcited by 37
  20. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 36
  21. 2022SPDY: Accurate Pruning with Speedup Guaranteescited by 34
  22. 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
  23. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  24. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkitcited by 32
  25. 2023FP8 versus INT8 for efficient deep learning inferencecited by 31
  26. 20241.58-bit FLUXcited by 31
  27. 2024LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detectioncited by 29
  28. 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 29
  29. 2021Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formatscited by 26
  30. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26
  31. 2024DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compactioncited by 26
  32. 2024Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniquescited by 24
  33. 2024Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversationscited by 24
  34. 2023A Practical Mixed Precision Algorithm for Post-Training Quantizationcited by 23
  35. 2024Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Modelscited by 23
  36. 2023Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxingcited by 21
  37. 2024Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMscited by 21
  38. 2025PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Casescited by 21

50 most cited of 453 All 453 (JSON)