mlsanityData to 7 Oct 2026

Foundations of Large Language Model Compression -- Part 1: Weight Quantization

arXiv 2409.02026 ↗First posted 3 Sep 2024cs.LG cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

In recent years, compression of large language models (LLMs) has emerged as an important problem to enable language model deployment on resource-constrained devices, reduce computational costs, and mitigate the environmental footprint of large-scale AI infrastructure. In this paper, we lay down the foundation for LLM quantization from a convex optimization perspective and propose a quantization technique that builds on this foundation for optimum quantization outcomes. Our quantization framework, CVXQ, scales to models containing hundreds of billions of weight parameters and provides users with the flexibility to compress models to any specified model size, post-training. A reference implementation of CVXQ can be obtained from github.com/seannz/cvxq.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 3 Sep 2024.

Builds on

34 of 47 references.

References · 34

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  6. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  7. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  8. 2019Learned Step Size Quantizationcited by 604
  9. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  10. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  11. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  12. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
22 more
  1. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  2. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  3. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  4. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  5. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  6. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  7. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  8. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  9. 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 189
  10. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 150
  11. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  12. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 113
  13. 2019Loss Aware Post-training Quantizationcited by 94
  14. 2020Generative Low-bitwidth Data Free Quantizationcited by 87
  15. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
  16. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 57
  17. 2021Qimera: Data-free Quantization with Synthetic Boundary Supporting Samplescited by 39
  18. 2024APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Modelscited by 33
  19. 2018Loss-aware Weight Quantization of Deep Networkscited by 31
  20. 2019Adaptive Loss-aware Quantization for Multi-bit Networkscited by 29
  21. 2023Adaptive Data-Free Quantizationcited by 26
  22. 2020Transform Quantization for CNN (Convolutional Neural Network) Compressioncited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025On Accelerating Edge AI: Optimizing Resource-Constrained Environmentscited by 5
  2. 2026Contraction-Gauge Preconditioning for Quantized Matrix Multiplicationcited by 1