mlsanityData to 7 Oct 2026

QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks

arXiv 2402.04396 ↗First posted 6 Feb 2024cs.LG cs.AI cs.CLcited by 354

Not on the map. Why?

Cited by 44 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Post-training quantization (PTQ) reduces the memory footprint of LLMs by quantizing their weights to low-precision. In this work, we introduce QuIP#, a weight-only PTQ method that achieves state-of-the-art results in extreme compression regimes (≤ 4 bits per weight) using three novel techniques. First, QuIP# improves QuIP's (Chee et al., 2023) incoherence processing by using the randomized Hadamard transform, which is faster and has better theoretical properties. Second, QuIP# uses vector quantization to take advantage of the ball-shaped sub-Gaussian distribution that incoherent weights possess: specifically, we introduce a set of hardware-efficient codebooks based on the highly symmetric E8 lattice, which achieves the optimal 8-dimension unit ball packing. Third, QuIP# uses fine-tuning to improve fidelity to the original model. Our experiments show that QuIP# outperforms existing PTQ methods, enables new behaviors in PTQ scaling, and supports fast inference. Our code can be found at https://github.com/Cornell-RelaxML/quip-sharp.

Key facts. Cited by 354 papers in the map (data through 7 Oct 2026). Builds on 15 of its 33 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

15 of 33 references.

References · 15

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  4. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  5. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  7. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  8. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  9. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  10. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  11. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  12. 2023HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolutioncited by 187
3 more
  1. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 116
  2. 2023Training Transformers with 4-bit Integerscited by 58
  3. 2021Model Preserving Compression for Neural Networkscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 61; 2025: 121; 2026: 172.

354 papers.

Sort:

By area:

  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  2. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  3. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
  4. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  5. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  6. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 231
  7. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  8. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
  9. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 123
  10. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
  11. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 94
  12. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
38 more
  1. 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 77
  2. 2025Pretraining Large Language Models with NVFP4cited by 67
  3. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
  4. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
  5. 2024Palu: Compressing KV-Cache with Low-Rank Projectioncited by 59
  6. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
  7. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
  8. 2024Training-Free Activation Sparsity in Large Language Modelscited by 51
  9. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 44
  10. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains Morecited by 38
  11. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 36
  12. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 32
  13. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  14. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 30
  15. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 28
  16. 2024Fast Matrix Multiplications for Lookup Table-Quantized LLMscited by 27
  17. 2024MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantizationcited by 26
  18. 2025MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methodscited by 26
  19. 2024Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Modelscited by 25
  20. 2024Quamba: A Post-Training Quantization Recipe for Selective State Space Modelscited by 25
  21. 2025Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantizationcited by 24
  22. 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Modelscited by 22
  23. 2025Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Modelscited by 21
  24. 2025QuEST: Stable Training of LLMs with 1-Bit Weights and Activationscited by 21
  25. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 21
  26. 2024LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Gridcited by 20
  27. 2024Small Singular Values Matter: A Random Matrix Analysis of Transformer Modelscited by 20
  28. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 20
  29. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recoverycited by 20
  30. 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmcited by 18
  31. 2025Block Rotation is All You Need for MXFP4 Quantizationcited by 18
  32. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 17
  33. 2025NestQuant: Nested Lattice Quantization for Matrix Products and LLMscited by 16
  34. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  35. 2025Quantized Visual Geometry Grounded Transformercited by 16
  36. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 16
  37. 2025SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Modelscited by 15
  38. 2025QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Modelscited by 15

50 most cited of 354 All 354 (JSON)