mlsanityData to 14 Oct 2026

Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Models

arXiv 2506.03781 ↗First posted 4 Jun 2025cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

How can we quantize large language models while preserving accuracy? Quantization is essential for deploying large language models (LLMs) efficiently. Binary-coding quantization (BCQ) and uniform quantization (UQ) are promising quantization schemes that have strong expressiveness and optimizability, respectively. However, neither scheme leverages both advantages. In this paper, we propose UniQuanF (Unified Quantization with Flexible Mapping), an accurate quantization method for LLMs. UniQuanF harnesses both strong expressiveness and optimizability by unifying the flexible mapping technique in UQ and non-uniform quantization levels of BCQ. We propose unified initialization, and local and periodic mapping techniques to optimize the parameters in UniQuanF precisely. After optimization, our unification theorem removes computational and memory overhead, allowing us to utilize the superior accuracy of UniQuanF without extra deployment costs induced by the unification. Experimental results demonstrate that UniQuanF outperforms existing UQ and BCQ methods, achieving up to 4.60% higher accuracy on GSM8K benchmark.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

29 of 49 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  12. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
17 more
  1. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  3. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  4. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  5. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  6. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 383
  7. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  8. 2015Deep Learning with Limited Numerical Precisioncited by 376
  9. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  10. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 105
  11. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 48
  12. 2018Alternating Multi-bit Quantization for Recurrent Neural Networkscited by 31
  13. 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 30
  14. 2022AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Modelscited by 23
  15. 2024A Comprehensive Survey of Compression Algorithms for Language Modelscited by 15
  16. 2025Zero-shot Quantization: A Comprehensive Surveycited by 8
  17. 2020AUBER: Automated BERT Regularizationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Prune-then-Quantize or Quantize-then-Prune? Understanding the Impact of Compression Order in Joint Model Compressioncited by 3
  2. 2026LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMscited by 2
  3. 2026LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projectioncited by 2
  4. 2026HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioningcited by 1
  5. 2026Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023PB-LLM: Partially Binarized Large Language Models
  2. 2026Statistically-Lossless Quantization of Large Language Models
  3. 2025Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retraining
  4. 2024LCQ: Low-Rank Codebook based Quantization for Large Language Models
  5. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
  6. 2024CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compression
  7. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
  8. 2024ARB-LLM: Alternating Refined Binarizations for Large Language Models