mlsanityData to 14 Oct 2026

SpinQuant: LLM quantization with learned rotations

arXiv 2405.16406 ↗First posted 26 May 2024cs.LG cs.AI cs.CLcited by 471

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

Post-training quantization (PTQ) techniques applied to weights, activations, and the KV cache greatly reduce memory usage, latency, and power consumption of Large Language Models (LLMs), but may lead to large quantization errors when outliers are present. Rotating activation or weight matrices helps remove outliers and benefits quantization. In this work, we identify a collection of applicable rotation parameterizations that lead to identical outputs in full-precision Transformer architectures while enhancing quantization accuracy. In addition, we find that some random rotations lead to much better quantization than others, with an up to 13 points difference in downstream zero-shot reasoning performance. As a result, we propose SpinQuant, a novel approach that incorporates learned rotation matrices for optimal quantized network accuracy. With 4-bit quantization of weight, activation, and KV-cache, SpinQuant narrows the accuracy gap on zero-shot reasoning tasks with full precision to merely 2.9 points on the LLaMA-2 7B model, surpassing LLM-QAT by 19.1 points and SmoothQuant by 25.0 points. Furthermore, SpinQuant also outperforms concurrent work QuaRot, which applies random rotations to remove outliers. In particular, for LLaMA-3 8B models that are hard to quantize, SpinQuant reduces the gap to full precision by up to 45.1% relative to QuaRot. Code is available at https://github.com/facebookresearch/SpinQuant.

Key facts. Cited by 471 papers in the map (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 39 of its 52 references that are in the map. First posted on arXiv 26 May 2024.

Builds on

39 of 52 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar
  2. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Serving29 Oct

Other references · 37

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  9. 2023Mistral 7Bcited by 3,164
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
25 more
  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  7. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  8. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  9. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  10. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  11. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  12. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  13. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  14. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  15. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  16. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  17. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  18. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  19. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
  20. 2019Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Groupcited by 134
  21. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 82
  22. 2020Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transformcited by 80
  23. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 69
  24. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 64
  25. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 38; 2025: 171; 2026: 262.

471 papers.

Sort:

By area:

  1. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 170
  2. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  3. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 131
  4. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 115
  5. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  6. 2024Palu: Compressing KV-Cache with Low-Rank Projectioncited by 64
  7. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 60
  8. 2025Optimizing Large Language Model Training Using FP4 Quantizationcited by 56
  9. 2024The Super Weight in Large Language Modelscited by 52
  10. 2025RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotationscited by 46
  11. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 46
  12. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Modelscited by 45
38 more
  1. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 41
  2. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inferencecited by 40
  3. 2024SAM Decoding: Speculative Decoding via Suffix Automatoncited by 39
  4. 2024BitsFusion: 1.99 bits Weight Quantization of Diffusion Modelcited by 37
  5. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 36
  6. 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 35
  7. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 32
  8. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 32
  9. 2024A Survey of Small Language Modelscited by 31
  10. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 28
  11. 2025Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantizationcited by 28
  12. 2026QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantizationcited by 28
  13. 2025TorchAO: PyTorch-Native Training-to-Serving Model Optimizationcited by 26
  14. 2024Towards Accurate Post-Training Quantization of Vision Transformers via Error Reductioncited by 23
  15. 2025QuEST: Stable Training of LLMs with 1-Bit Weights and Activationscited by 22
  16. 2025OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Modelscited by 22
  17. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recoverycited by 22
  18. 2024Pushing the Limits of Large Language Model Quantization via the Linearity Theoremcited by 21
  19. 2024MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Modelscited by 21
  20. 2025GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuningcited by 21
  21. 2025An Empirical Study of Qwen3 Quantizationcited by 21
  22. 2025Block Rotation is All You Need for MXFP4 Quantizationcited by 20
  23. 2024AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inferencecited by 19
  24. 2025BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMscited by 18
  25. 2025INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formatscited by 18
  26. 2025Scaling Law for Quantization-Aware Trainingcited by 17
  27. 2025DartQuant: Efficient Rotational Distribution Calibration for LLM Quantizationcited by 17
  28. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 17
  29. 2024BitNet a4.8: 4-bit Activations for 1-bit LLMscited by 16
  30. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 16
  31. 2025Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Modelscited by 16
  32. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  33. 2025Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronoscited by 16
  34. 2025Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamicscited by 16
  35. 2025QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Modelscited by 16
  36. 2025HALO: Hadamard-Assisted Lower-Precision Optimization for LLMscited by 15
  37. 2025A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clonecited by 15
  38. 2025DVD-Quant: Data-free Video Diffusion Transformers Quantizationcited by 15

50 most cited of 471 All 471 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
  2. 2024LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid
  3. 2023SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM
  4. 2024DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotation
  5. 2025FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization
  6. 2026TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
  7. 2026RUQuant: Towards Refining Uniform Quantization for Large Language Models
  8. 2024FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantization