mlsanityData to 7 Oct 2026

BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

arXiv 2402.04291 ↗First posted 6 Feb 2024cs.LG cs.AI cs.CLcited by 137

Not on the map. Why?

Abstract (arXiv)

Pretrained large language models (LLMs) exhibit exceptional general language processing capabilities but come with significant demands on memory and computational resources. As a powerful compression technology, binarization can extremely reduce model weights to a mere 1 bit, lowering the expensive computation and memory requirements. However, existing quantization techniques fall short of maintaining LLM performance under ultra-low bit-widths. In response to this challenge, we present BiLLM, a groundbreaking 1-bit post-training quantization scheme tailored for pretrained LLMs. Based on the weight distribution of LLMs, BiLLM first identifies and structurally selects salient weights, and minimizes the compression loss through an effective binary residual approximation strategy. Moreover, considering the bell-shaped distribution of the non-salient weights, we propose an optimal splitting search to group and binarize them accurately. BiLLM achieving for the first time high-accuracy inference (e.g. 8.41 perplexity on LLaMA2-70B) with only 1.08-bit weights across various LLMs families and evaluation metrics, outperforms SOTA quantization methods of LLM by significant margins. Moreover, BiLLM enables the binarization process of the LLM with 7 billion weights within 0.5 hours on a single GPU, demonstrating satisfactory time efficiency. Our code is available at https://github.com/Aaronhuang-778/BiLLM.

Key facts. Cited by 137 papers in the map (data through 7 Oct 2026). Builds on 41 of its 56 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

41 of 56 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  11. 2016Pointer Sentinel Mixture Modelscited by 2,992
  12. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
29 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  2. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  3. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  5. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  10. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  11. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  12. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  13. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  14. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  15. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  16. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  17. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  18. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  19. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  20. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  21. 2023A Survey on Model Compression for Large Language Modelscited by 263
  22. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
  23. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  24. 2023PB-LLM: Partially Binarized Large Language Modelscited by 80
  25. 2020Post-Training Piecewise Linear Quantization for Deep Neural Networkscited by 77
  26. 2019Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimizationcited by 58
  27. 2018Value-aware Quantization for Training and Inference of Neural Networkscited by 49
  28. 2017Performance Guaranteed Network Acceleration via High-Order Residual Quantizationcited by 24
  29. 2018SYQ: Learning Symmetric Quantization For Efficient Deep Neural Networkscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 37; 2025: 57; 2026: 43.

137 papers.

Sort:

By area:

  1. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 231
  2. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  3. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  4. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  5. 2024Scaling Laws for Precisioncited by 92
  6. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
  7. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 63
  8. 2025SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compressioncited by 63
  9. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
  10. 2024Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Modelscited by 60
  11. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
  12. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
38 more
  1. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 44
  2. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains Morecited by 38
  3. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  4. 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
  5. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  6. 2024ARB-LLM: Alternating Refined Binarizations for Large Language Modelscited by 30
  7. 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 27
  8. 2024CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaborationcited by 25
  9. 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Modelscited by 22
  10. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  11. 2024ParallelSpec: Parallel Drafter for Efficient Speculative Decodingcited by 20
  12. 2025An Empirical Study of Qwen3 Quantizationcited by 20
  13. 2025A Survey on Efficient Vision-Language Modelscited by 19
  14. 2026Flow caching for autoregressive video generationcited by 19
  15. 2024Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokenscited by 18
  16. 2025EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Modelscited by 18
  17. 2025Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Modelscited by 17
  18. 2025DarwinLM: Evolutionary Structured Pruning of Large Language Modelscited by 16
  19. 2025AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Modelscited by 15
  20. 2024Mixture of Scales: Memory-Efficient Token-Adaptive Binarization for Large Language Modelscited by 14
  21. 2024Beware of Calibration Data for Pruning Large Language Modelscited by 14
  22. 2025QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMscited by 13
  23. 2024FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillationcited by 12
  24. 2024BinaryDM: Accurate Weight Binarization for Efficient Diffusion Modelscited by 11
  25. 2024Channel-Wise Mixed-Precision Quantization for Large Language Modelscited by 11
  26. 2024BiDM: Pushing the Limit of Quantization for Diffusion Modelscited by 11
  27. 2025LittleBit: Ultra Low-Bit Quantization via Latent Factorizationcited by 11
  28. 2025Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Modelscited by 11
  29. 2024Bi-Mamba: Towards Accurate 1-Bit State Space Modelscited by 10
  30. 2024Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Maskingcited by 10
  31. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
  32. 2024INT-FlashAttention: Enabling Flash Attention for INT8 Quantizationcited by 9
  33. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Modelscited by 9
  34. 2024OAC: Output-adaptive Calibration for Accurate Post-training Quantizationcited by 7
  35. 2024Mixture of Experts with Mixture of Precisions for Tuning Quality of Servicecited by 7
  36. 2024Designing Large Foundation Models for Efficient Training and Inference: A Surveycited by 7
  37. 2024CRVQ: Channel-Relaxed Vector Quantization for Extreme Compression of LLMscited by 7
  38. 2025Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs: Comprehensive Analysis and Defensecited by 7

50 most cited of 137 All 137 (JSON)