mlsanityData to 14 Oct 2026

On the Expressive Power of Weight Quantization in Large Language Models

arXiv 2606.22249 ↗First posted 20 Jun 2026cs.LG cs.AIcited by 0

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

In recent years, weight quantization that encodes the learnable parameters of large language models in an n-bit format has garnered significant attention due to its potential for model compression and inference acceleration. Many practical techniques have been developed; however, the theoretical understanding of many aspects, especially the approximation and degradation of expressive power as the number of quantization bits decreases, remains unclear. In this paper, we provide a theoretical investigation into the expressive capability of large language models relative to the number of quantization bits. We argue that 1.58-bit is the limiting precision for weight quantization by establishing the universal approximation and expressive collapse properties of weight-quantized models with respect to the number of quantization bits. Additionally, we confirm that weight quantization leads to expressive degradation, in which the expressive capacity of weight-quantized models degrades polynomially as the number of quantization bits decreases. These theoretical findings provide a solid foundation for advancing weight quantization in the context of scaling laws and shed insights for future research in model compression and inference acceleration.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 35 of its 47 references that are in the map. First posted on arXiv 20 Jun 2026.

Builds on

35 of 47 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  8. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  9. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  10. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  11. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  12. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
23 more
  1. 2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 406
  2. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  3. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  4. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  5. 2016Ternary Weight Networkscited by 204
  6. 2019Universal Approximation with Deep Narrow Networkscited by 200
  7. 2017Training Quantized Nets: A Deeper Understandingcited by 75
  8. 2018A Survey on Methods and Theories of Quantized Neural Networkscited by 75
  9. 2018SqueezeNext: Hardware-Aware Neural Network Designcited by 61
  10. 2020Searching for Low-Bit Weights in Quantized Neural Networkscited by 44
  11. 2025BitNet b1.58 2B4T Technical Reportcited by 38
  12. 2023Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edgecited by 33
  13. 2020The universal approximation theorem for complex-valued neural networkscited by 28
  14. 2017The High-Dimensional Geometry of Binary Neural Networkscited by 22
  15. 2024Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokenscited by 18
  16. 2018On the Universal Approximability and Complexity Bounds of Quantized ReLU Neural Networkscited by 12
  17. 2022Approximation speed of quantized vs. unquantized ReLU neural networks and beyondcited by 8
  18. 2019Ternary Hybrid Neural-Tree Networks for Highly Constrained IoT Applicationscited by 4
  19. 2024Feed-Forward Neural Networks as a Mixed-Integer Programcited by 4
  20. 2025TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledgecited by 4
  21. 2021Universal Approximation Theorems of Fully Connected Binarized Neural Networkscited by 2
  22. 2025A unified framework for establishing the universal approximation of transformer-type architecturescited by 2
  23. 2024Efficient Ternary Weight Embedding Model: Bridging Scalability and Performancecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Compression Scaling Laws:Unifying Sparsity and Quantization
  2. 2025Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size
  3. 2024What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation
  4. 2026Quantization Degradation in Large Language Models: A Signal-Noise Perspective
  5. 2026Statistically-Lossless Quantization of Large Language Models
  6. 2024Foundations of Large Language Model Compression -- Part 1: Weight Quantization
  7. 2024Scaling Laws for Post Training Quantized Large Language Models
  8. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Models