mlsanityData to 14 Oct 2026

Extreme Compression of Large Language Models via Additive Quantization

arXiv 2401.06118 ↗First posted 11 Jan 2024cs.LG cs.CLcited by 232

Not on the map. Why?

Cited by 22 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The emergence of accurate open large language models (LLMs) has led to a race towards performant quantization techniques which can enable their execution on end-user devices. In this paper, we revisit the problem of "extreme" LLM compression-defined as targeting extremely low bit counts, such as 2 to 3 bits per parameter-from the point of view of classic methods in Multi-Codebook Quantization (MCQ). Our algorithm, called AQLM, generalizes the classic Additive Quantization (AQ) approach for information retrieval to advance the state-of-the-art in LLM compression, via two innovations: 1) learned additive quantization of weight matrices in input-adaptive fashion, and 2) joint optimization of codebook parameters across each transformer blocks. Broadly, AQLM is the first scheme that is Pareto optimal in terms of accuracy-vs-model-size when compressing to less than 3 bits per parameter, and significantly improves upon all known schemes in the extreme compression (2bit) regime. In addition, AQLM is practical: we provide fast GPU and CPU implementations of AQLM for token generation, which enable us to match or outperform optimized FP16 implementations for speed, while executing in a much smaller memory footprint.

Key facts. Cited by 232 papers in the map (data through 14 Oct 2026). Builds on 36 of its 66 references that are in the map. First posted on arXiv 11 Jan 2024.

Builds on

36 of 66 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  12. 2023Mistral 7Bcited by 3,164
24 more
  1. 2016Pointer Sentinel Mixture Modelscited by 3,096
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  4. 2024Mixtral of Expertscited by 1,823
  5. 2020GLU Variants Improve Transformercited by 1,804
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  7. 2019Root Mean Square Layer Normalizationcited by 1,480
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  9. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  10. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  11. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  12. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  13. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 508
  14. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  15. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  16. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  17. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  18. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  19. 2023QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Modelscited by 42
  20. 2021Multiplying Matrices Without Multiplyingcited by 30
  21. 2015Quantization based Fast Inner Product Searchcited by 29
  22. 2017Performance Guaranteed Network Acceleration via High-Order Residual Quantizationcited by 26
  23. 2017Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networkscited by 21
  24. 2022Look-ups are not (yet) all you need for deep learning inferencecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 60; 2025: 74; 2026: 98.

232 papers.

Sort:

By area:

  1. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  2. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  3. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  4. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  5. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  6. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  7. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  8. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 82
  9. 2024Exploiting LLM Quantizationcited by 65
  10. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  11. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61
  12. 2024Compressing Large Language Models using Low Rank and Low Precision Decompositioncited by 59
38 more
  1. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 57
  2. 2024Training-Free Activation Sparsity in Large Language Modelscited by 55
  3. 2024MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Modelscited by 52
  4. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 46
  5. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains Morecited by 41
  6. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 41
  7. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 36
  8. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkitcited by 33
  9. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 32
  10. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 32
  11. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 31
  12. 2024Fast Matrix Multiplications for Lookup Table-Quantized LLMscited by 29
  13. 2025Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Modelscited by 24
  14. 2024Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularizationcited by 23
  15. 2025Model-Preserving Adaptive Roundingcited by 23
  16. 2024LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Gridcited by 22
  17. 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Modelscited by 22
  18. 2024Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scalecited by 21
  19. 2024Small Singular Values Matter: A Random Matrix Analysis of Transformer Modelscited by 21
  20. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 21
  21. 2025R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inferencecited by 21
  22. 2024OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decompositioncited by 20
  23. 2024EvoPress: Accurate Dynamic Model Compression via Evolutionary Searchcited by 20
  24. 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmcited by 20
  25. 2024BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillationcited by 19
  26. 2024Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokenscited by 18
  27. 2025Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Modelscited by 18
  28. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 16
  29. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  30. 2025Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronoscited by 16
  31. 2025HALO: Hadamard-Assisted Lower-Precision Optimization for LLMscited by 15
  32. 2024TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Taskscited by 14
  33. 2025ORANSight-2.0: Foundational LLMs for O-RANcited by 13
  34. 2024LoQT: Low-Rank Adapters for Quantized Pretrainingcited by 12
  35. 2024BoA: Attention-aware Post-training Quantization without Backpropagationcited by 12
  36. 2024AXIS: Efficient Human-Agent-Computer Interaction with API-First LLM-Based Agentscited by 12
  37. 2024Bi-Mamba: Towards Accurate 1-Bit State Space Modelscited by 12
  38. 2025KurTail : Kurtosis-based LLM Quantizationcited by 12

50 most cited of 232 All 232 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LCQ: Low-Rank Codebook based Quantization for Large Language Models
  2. 2025When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models
  3. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simple
  4. 2025LLM Compression: How Far Can We Go in Balancing Size and Performance?
  5. 2023PB-LLM: Partially Binarized Large Language Models
  6. 2023Norm Tweaking: High-performance Low-bit Quantization of Large Language Models
  7. 2025AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMs
  8. 2024ARB-LLM: Alternating Refined Binarizations for Large Language Models