mlsanityData to 14 Oct 2026

Radio: Rate-Distortion Optimization for Large Language Model Compression

arXiv 2505.03031 ↗First posted 5 May 2025cs.LG cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

In recent years, the compression of large language models (LLMs) has emerged as a key problem in facilitating LLM deployment on resource-limited devices, reducing compute costs, and mitigating the environmental footprint due to large-scale AI infrastructure. Here, we establish the foundations of LLM quantization from a rate-distortion theory perspective and propose a quantization technique based on simple rate-distortion optimization. Our technique scales to models containing hundreds of billions of weight parameters and offers users the flexibility to compress models, post-training, to a model size or accuracy specified by the user.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 5 May 2025.

Builds on

39 of 52 references.

References · 39

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  10. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  11. 2019Learned Step Size Quantizationcited by 628
  12. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
27 more
  1. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  2. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  3. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  4. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  5. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  6. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  7. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  8. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  9. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  10. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  11. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  12. 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 195
  13. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  14. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  15. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  16. 2019Loss Aware Post-training Quantizationcited by 97
  17. 2020Generative Low-bitwidth Data Free Quantizationcited by 88
  18. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 82
  19. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  20. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simplecited by 62
  21. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 61
  22. 2021Qimera: Data-free Quantization with Synthetic Boundary Supporting Samplescited by 40
  23. 2024APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Modelscited by 34
  24. 2018Loss-aware Weight Quantization of Deep Networkscited by 31
  25. 2019Adaptive Loss-aware Quantization for Multi-bit Networkscited by 30
  26. 2023Adaptive Data-Free Quantizationcited by 26
  27. 2020Transform Quantization for CNN (Convolutional Neural Network) Compressioncited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Latent Semantic Manifolds in Large Language Modelscited by 5
  2. 2026RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theorycited by 3
  3. 2025Optimal Formats for Weight Quantisationcited by 1
  4. 2026ADKO: Agentic Decentralized Knowledge Optimizationcited by 0
  5. 2026DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Foundations of Large Language Model Compression -- Part 1: Weight Quantization
  2. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
  3. 2025When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models
  4. 2025Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques
  5. 2023A Survey on Model Compression for Large Language Models
  6. 2024On the Compressibility of Quantized Large Language Models
  7. 2024Continuous Approximations for Improving Quantization Aware Training of LLMs
  8. 2025Semantic Retention and Extreme Compression in LLMs: Can We Have Both?