mlsanityData to 7 Oct 2026

Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation

arXiv 2004.09602 ↗First posted 20 Apr 2020cs.LG stat.MLcited by 183

Not on the map. Why?

Abstract (arXiv)

Quantization techniques can reduce the size of Deep Neural Networks and improve inference latency and throughput by taking advantage of high throughput integer instructions. In this paper we review the mathematical aspects of quantization parameters and evaluate their choices on a wide range of neural network models for different application domains, including vision, speech, and language. We focus on quantization techniques that are amenable to acceleration by processors with high-throughput integer math pipelines. We also present a workflow for 8-bit quantization that is able to maintain accuracy within 1% of the floating-point baseline on all networks studied, including models that are more difficult to quantize, such as MobileNets and BERT-large.

Key facts. Cited by 183 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 30; 2023: 35; 2024: 40; 2025: 32; 2026: 23.

183 papers.

Sort:

By area:

  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  3. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  4. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  5. 2021Knowledge distillation: A good teacher is patient and consistentcited by 255
  6. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  7. 2021Accelerating Sparse Deep Neural Networkscited by 213
  8. 2022FP8 Formats for Deep Learningcited by 187
  9. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  10. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  11. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 93
  12. 2024Scaling Laws for Precisioncited by 92
38 more
  1. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 62
  2. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 59
  3. 2024Transformers Provably Solve Parity Efficiently with Chain of Thoughtcited by 49
  4. 2023Intriguing Properties of Quantization at Scalecited by 48
  5. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 46
  6. 2023ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formatscited by 44
  7. 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
  8. 2024A Comprehensive Study on Quantization Techniques for Large Language Modelscited by 34
  9. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  10. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  11. 2021Pareto-Optimal Quantized ResNet Is Mostly 4-bitcited by 26
  12. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26
  13. 2022Optimal Clipping and Magnitude-aware Differentiation for Improved Quantization-aware Trainingcited by 26
  14. 2022Who Says Elephants Can't Run: Bringing Large Scale MoE Models into Cloud Scale Productioncited by 24
  15. 2023Boost Vision Transformer with GPU-Friendly Sparsity and Quantizationcited by 20
  16. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recoverycited by 20
  17. 2024I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Modelscited by 19
  18. 2023Integer or Floating Point? New Outlooks for Low-Bit Quantization on Large Language Modelscited by 18
  19. 2022Efficient Quantized Sparse Matrix Operations on Tensor Corescited by 16
  20. 2023FPTQ: Fine-grained Post-Training Quantization for Large Language Modelscited by 16
  21. 2024A Comprehensive Survey of Compression Algorithms for Language Modelscited by 15
  22. 2024ESPACE: Dimensionality Reduction of Activations for Model Compressioncited by 15
  23. 2024QVD: Post-training Quantization for Video Diffusion Modelscited by 14
  24. 2023Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?cited by 13
  25. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Surveycited by 13
  26. 2025Adaptive Semantic Token Communication for Transformer-based Edge Inferencecited by 13
  27. 2021Improving the Efficiency of Transformers for Resource-Constrained Devicescited by 12
  28. 2021Reinforcement Learning for Datacenter Congestion Controlcited by 11
  29. 2023A Speed Odyssey for Deployable Quantization of LLMscited by 10
  30. 2024Adaptive Semantic Token Selection for AI-native Goal-oriented Communicationscited by 10
  31. 2021HPTQ: Hardware-Friendly Post Training Quantizationcited by 9
  32. 2021Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniquescited by 9
  33. 2022What Do Compressed Multilingual Machine Translation Models Forget?cited by 9
  34. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  35. 2024Q-S5: Towards Quantized State Space Modelscited by 9
  36. 2021Deployment of Deep Neural Networks for Object Detection on Edge AI Devices with Runtime Optimizationcited by 8
  37. 2025FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Modelscited by 8
  38. 2021Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scalecited by 7

50 most cited of 183 All 183 (JSON)