mlsanityData to 14 Oct 2026

ICQuant: Index Coding enables Low-bit LLM Quantization

arXiv 2505.00850 ↗First posted 1 May 2025cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

The rapid deployment of Large Language Models (LLMs) highlights the need for efficient low-bit post-training quantization (PTQ), due to their high memory costs. A key challenge in weight quantization is the presence of outliers, which inflate quantization ranges and lead to large errors. While a number of outlier suppression techniques have been proposed, they either: fail to effectively shrink the quantization range, or incur (relatively) high bit overhead. In this paper, we present ICQuant, a novel framework that leverages outlier statistics to design an efficient index coding scheme for outlier-aware weight-only quantization. Compared to existing outlier suppression techniques requiring ≈ 1 bit overhead to halve the quantization range, ICQuant requires only ≈ 0.3 bits; a significant saving in extreme compression regimes (e.g., 2-3 bits per weight). ICQuant can be used on top of any existing quantizers to eliminate outliers, improving the quantization quality. Using just 2.3 bits per weight and simple scalar quantizers, ICQuant improves the zero-shot accuracy of the 2-bit Llama3-70B model by up to 130% and 150% relative to QTIP and QuIP#; and it achieves comparable performance to the best-known fine-tuned quantizer (PV-tuning) without fine-tuning.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 1 May 2025.

Builds on

27 of 35 references.

References · 27

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  10. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  12. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
15 more
  1. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  2. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  3. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  4. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  5. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  6. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  7. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  8. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  9. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  10. 2024Massive Activations in Large Language Modelscited by 294
  11. 2023A Survey on Model Compression for Large Language Modelscited by 269
  12. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  13. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  14. 2024BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillationcited by 84
  15. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 5.

9 papers.

  1. 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmcited by 20
  2. 2026ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMscited by 3
  3. 2026ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Modelscited by 3
  4. 2025ARMOR: High-Performance Semi-Structured Pruning via Adaptive Matrix Factorizationcited by 2
  5. 2026SNIP: An Adaptive Mixed Precision Framework for Subbyte Large Language Model Trainingcited by 2
  6. 2026UniSVQ: 2-bit Unified Scalar-Vector Quantizationcited by 2
  7. 2025CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Roundingcited by 1
  8. 2025It Takes a Good Model to Train a Good Model: Generalized Gaussian Priors for Optimized LLMscited by 0
  9. 2026BiSCo-LLM: Lookup-Free Binary Spherical Coding for Extreme Low-Bit Large Language Model Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models
  2. 2026OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantization
  3. 2026MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
  4. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs
  5. 2024EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
  6. 2024PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization
  7. 2024OutlierTune: Efficient Channel-Wise Quantization for Large Language Models
  8. 2026NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models