mlsanityData to 7 Oct 2026

A Survey of Quantization Methods for Efficient Neural Network Inference

arXiv 2103.13630 ↗First posted 25 Mar 2021cs.CVcited by 798

Not on the map. Why?

Cited by 21 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As soon as abstract mathematical computations were adapted to computation on digital computers, the problem of efficient representation, manipulation, and communication of the numerical values in those computations arose. Strongly related to the problem of numerical representation is the problem of quantization: in what manner should a set of continuous real-valued numbers be distributed over a fixed discrete set of numbers to minimize the number of bits required and also to maximize the accuracy of the attendant computations? This perennial problem of quantization is particularly relevant whenever memory and/or computational resources are severely restricted, and it has come to the forefront in recent years due to the remarkable performance of Neural Network models in computer vision, natural language processing, and related areas. Moving from floating-point representations to low-precision fixed integer values represented in four bits or less holds the potential to reduce the memory footprint and latency by a factor of 16x; and, in fact, reductions of 4x to 8x are often realized in practice in these applications. Thus, it is not surprising that quantization has emerged recently as an important and very active sub-area of research in the efficient implementation of computations associated with Neural Networks. In this article, we survey approaches to the problem of quantizing the numerical values in deep Neural Network computations, covering the advantages/disadvantages of current methods. With this survey and its organization, we hope to have presented a useful snapshot of the current research in quantization for Neural Networks and to have given an intelligent organization to ease the evaluation of future research in this area.

Key facts. Cited by 798 papers in the map (data through 7 Oct 2026). Builds on 198 of its 302 references that are in the map. First posted on arXiv 25 Mar 2021.

Builds on

198 of 302 references.

References · 198

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2016Densely Connected Convolutional Networkscited by 6,691
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  11. 2016Layer Normalizationcited by 5,579
  12. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
38 more
  1. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  7. 2019Searching for MobileNetV3cited by 1,984
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  9. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  10. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  11. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  12. 2017Mixed Precision Trainingcited by 1,135
  13. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  14. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  15. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  16. 2019Relational Knowledge Distillationcited by 873
  17. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  18. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  19. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  20. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  21. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  22. 2019Learned Step Size Quantizationcited by 604
  23. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  24. 2020What is the State of Neural Network Pruning?cited by 578
  25. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  26. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  27. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  28. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  29. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  30. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  31. 2019The State of Sparsity in Deep Neural Networkscited by 471
  32. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  33. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  34. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  35. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  36. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  37. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  38. 2015Deep Learning with Limited Numerical Precisioncited by 368

50 most cited of 198 All 198 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 33; 2022: 107; 2023: 147; 2024: 196; 2025: 178; 2026: 137.

798 papers.

Sort:

By area:

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  3. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  4. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  5. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  6. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
  7. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
  8. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  9. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  10. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  11. 2022Matryoshka Representation Learningcited by 268
  12. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 266
38 more
  1. 2023A Survey on Model Compression for Large Language Modelscited by 263
  2. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  3. 2023Large Language Models in Finance: A Surveycited by 223
  4. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  5. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  6. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
  7. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  8. 2021Learned Token Pruning for Transformerscited by 138
  9. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  10. 2022PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Modelcited by 133
  11. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  12. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  13. 2023Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modelingcited by 94
  14. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 93
  15. 2024AI and Memory Wallcited by 89
  16. 2023Fast Inference of Mixture-of-Experts Language Models with Offloadingcited by 88
  17. 2022FP8 Quantization: The Power of the Exponentcited by 87
  18. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 87
  19. 2024Asymmetry in Low-Rank Adapters of Foundation Modelscited by 81
  20. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  21. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 78
  22. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
  23. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
  24. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 74
  25. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  26. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 71
  27. 2022Compressing Volumetric Radiance Fields to 1 MBcited by 66
  28. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  29. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  30. 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 59
  31. 2023Stateful Large Language Model Serving with Pensievecited by 59
  32. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
  33. 2021AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networkscited by 58
  34. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 55
  35. 2024TinyAgent: Function Calling at the Edgecited by 55
  36. 2024Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMscited by 54
  37. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
  38. 2024MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformercited by 53

50 most cited of 798 All 798 (JSON)