mlsanityData to 7 Oct 2026

QuIP: 2-Bit Quantization of Large Language Models With Guarantees

arXiv 2307.13304 ↗First posted 25 Jul 2023cs.LG cs.CLcited by 397

Not on the map. Why?

Cited by 37 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This work studies post-training parameter quantization in large language models (LLMs). We introduce quantization with incoherence processing (QuIP), a new method based on the insight that quantization benefits from textitincoherent weight and Hessian matrices, i.e., from the weights being even in magnitude and the directions in which it is important to round them accurately being unaligned with the coordinate axes. QuIP consists of two steps: (1) an adaptive rounding procedure minimizing a quadratic proxy objective; (2) efficient pre- and post-processing that ensures weight and Hessian incoherence via multiplication by random orthogonal matrices. We complement QuIP with the first theoretical analysis for an LLM-scale quantization algorithm, and show that our theory also applies to an existing method, OPTQ. Empirically, we find that our incoherence preprocessing improves several existing quantization algorithms and yields the first LLM quantization methods that produce viable results using only two bits per weight. Our code can be found at https://github.com/Cornell-RelaxML/QuIP.

Key facts. Cited by 397 papers in the map (data through 7 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

25 of 35 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  6. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  7. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  8. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  9. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  10. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  11. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  12. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
13 more
  1. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  2. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  3. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  4. 2021Post-Training Quantization for Vision Transformercited by 274
  5. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  6. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  7. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
  8. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
  9. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 98
  10. 2022NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformerscited by 59
  11. 2018A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Datasetcited by 54
  12. 2014Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problemscited by 25
  13. 2020A Greedy Algorithm for Quantizing Neural Networkscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 113; 2025: 129; 2026: 140.

397 papers.

Sort:

By area:

  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  2. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  3. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  4. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  5. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  6. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  7. 2023A Survey on Model Compression for Large Language Modelscited by 263
  8. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  9. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  10. 2023Efficient Large Language Models: A Surveycited by 183
  11. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
  12. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
38 more
  1. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  2. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 137
  3. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 123
  4. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
  5. 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 116
  6. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
  7. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 110
  8. 2024PowerInfer-2: Fast Large Language Model Inference on a Smartphonecited by 101
  9. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 94
  10. 2024Scaling Laws for Precisioncited by 92
  11. 2023ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Modelscited by 91
  12. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 86
  13. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  14. 2023PB-LLM: Partially Binarized Large Language Modelscited by 80
  15. 2024BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillationcited by 80
  16. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  17. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains Morecited by 69
  18. 2023EdgeMoE: Empowering Sparse Large Language Models on Mobile Devicescited by 65
  19. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  20. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 63
  21. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  22. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
  23. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
  24. 2024Compressing Large Language Models using Low Rank and Low Precision Decompositioncited by 58
  25. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 54
  26. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
  27. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
  28. 2024The Super Weight in Large Language Modelscited by 51
  29. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 50
  30. 2024MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Modelscited by 48
  31. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Modelscited by 45
  32. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 44
  33. 2023QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Modelscited by 42
  34. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  35. 2025RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotationscited by 42
  36. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
  37. 2024BitDelta: Your Fine-Tune May Only Be Worth One Bitcited by 38
  38. 2025Quartet: Native FP4 Training Can Be Optimal for Large Language Modelscited by 37

50 most cited of 397 All 397 (JSON)