# Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques

arXiv: https://arxiv.org/abs/2411.06084 · HTML: https://mlsanity.com/paper/2411.06084/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 9 Nov 2024.

## Builds on (most cited first, 7 of 7)

- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks](https://mlsanity.com/paper/1908.05033/) (2019)
- [Trained Ternary Quantization](https://mlsanity.com/paper/1612.01064/) (2016)
- [Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model](https://mlsanity.com/paper/1906.00532/) (2019)
- [QT-DoG: Quantization-aware Training for Domain Generalization](https://mlsanity.com/paper/2410.06020/) (2024)
- [AMED: Automatic Mixed-Precision Quantization for Edge Devices](https://mlsanity.com/paper/2205.15437/) (2022)

## Built on by (most cited first, 10 of 10)

- [Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency](https://mlsanity.com/paper/2504.03360/) (2025)
- [Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models](https://mlsanity.com/paper/2508.18609/) (2025)
- [LegalEval-Q: A New Benchmark for The Quality Evaluation of LLM-Generated Legal Text](https://mlsanity.com/paper/2505.24826/) (2025)
- [Pushing the Envelope of LLM Inference with Ultra-Low-Bit Quantized Models](https://mlsanity.com/paper/2508.06753/) (2025)
- [A Layer Importance Metric for Quantization Accounting for the Speed-Quality Trade-off in Autoregressive Models](https://mlsanity.com/paper/2608.26926/) (2026)
- [SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Token](https://mlsanity.com/paper/2507.17618/) (2025)
- [LLM Compression: How Far Can We Go in Balancing Size and Performance?](https://mlsanity.com/paper/2508.11318/) (2025)
- [The Thermodynamic Costs of Simple Linear Regression](https://mlsanity.com/paper/2605.19195/) (2026)
- [A Method for Layer Bit-Width Allocation in LLM Quantization via Performance Maximization Under a Quality-Degradation Constraint](https://mlsanity.com/paper/2608.28003/) (2026)
- [Q-MINO: A Minimal-Norm Method for Quantization-Aware Training](https://mlsanity.com/paper/2610.00738/) (2026)

## Abstract (arXiv)

This paper presents a comprehensive analysis of quantization techniques for optimizing Large Language Models (LLMs), specifically focusing on Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT). Through empirical evaluation across models ranging from 10M to 1B parameters, we demonstrate that quantization can achieve up to 68% reduction in model size while maintaining performance within 6% of full-precision baselines when utilizing our proposed scaling factor γ. Our experiments show that INT8 quantization delivers a 40% reduction in computational cost and power consumption, while INT4 quantization further improves these metrics by 60%. We introduce a novel theoretical framework for mixed-precision quantization, deriving optimal bit allocation strategies based on layer sensitivity and weight variance. Hardware efficiency evaluations on edge devices reveal that our quantization approach enables up to 2.4x throughput improvement for INT8 and 3x for INT4, with 60% power reduction compared to full-precision models.