Resource-Efficient Language Models: Quantization for Fast and Accessible Inference
Not on the map. Why?
Abstract (arXiv)
Large language models have significantly advanced natural language processing, yet their heavy resource demands pose severe challenges regarding hardware accessibility and energy consumption. This paper presents a focused and high-level review of post-training quantization (PTQ) techniques designed to optimize the inference efficiency of LLMs by the end-user, including details on various quantization schemes, granularities, and trade-offs. The aim is to provide a balanced overview between the theory and applications of post-training quantization.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 60 references that are in the map. First posted on arXiv 13 May 2025.
Builds on
41 of 60 references.
References · 41
- 2017Attention Is All You Needcited by 59,932
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
29 more
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
- 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
- 2023Textbooks Are All You Needcited by 561
- 2021A White Paper on Neural Network Quantizationcited by 453
- 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
- 2015Deep Learning with Limited Numerical Precisioncited by 368
- 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
- 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
- 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
- 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
- 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
- 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 147
- 2020Training with Quantization Noise for Extreme Model Compressioncited by 145
- 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
- 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 137
- 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 125
- 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 110
- 2020GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inferencecited by 103
- 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
- 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 63
- 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 44
- 2024DB-LLM: Accurate Dual-Binarization for Efficient LLMscited by 35
- 2024A Comprehensive Study on Quantization Techniques for Large Language Modelscited by 34
- 2023FP8 versus INT8 for efficient deep learning inferencecited by 31
- 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
- 2023Democratizing LLMs: An Exploration of Cost-Performance Trade-offs in Self-Refined Open-Source Modelscited by 11
- 2024Exploring Quantization for Efficient Pre-Training of Transformer Language Modelscited by 10
- 2024Continuous Approximations for Improving Quantization Aware Training of LLMscited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.