mlsanityData to 14 Oct 2026

KurTail : Kurtosis-based LLM Quantization

arXiv 2503.01483 ↗First posted 3 Mar 2025cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

One of the challenges of quantizing a large language model (LLM) is the presence of outliers. Outliers often make uniform quantization schemes less effective, particularly in extreme cases such as 4-bit quantization. We introduce KurTail, a new post-training quantization (PTQ) scheme that leverages Kurtosis-based rotation to mitigate outliers in the activations of LLMs. Our method optimizes Kurtosis as a measure of tailedness. This approach enables the quantization of weights, activations, and the KV cache in 4 bits. We utilize layer-wise optimization, ensuring memory efficiency. KurTail outperforms existing quantization methods, offering a 13.3% boost in MMLU accuracy and a 15.5% drop in Wiki perplexity compared to QuaRot. It also outperforms SpinQuant with a 2.6% MMLU gain and reduces perplexity by 2.9%, all while reducing the training cost. For comparison, learning the rotation using SpinQuant for Llama3-70B requires at least four NVIDIA H100 80GB GPUs, whereas our method requires only a single GPU, making it a more accessible solution for consumer GPU.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

26 of 43 references.

References · 26

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2016Pointer Sentinel Mixture Modelscited by 3,096
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  11. 2024Qwen2 Technical Reportcited by 2,120
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
14 more
  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  3. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  4. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  5. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  6. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  7. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  8. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  9. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 179
  10. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  11. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  12. 2020Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transformcited by 80
  13. 2020Robust Quantization: One Model to Rule Them Allcited by 63
  14. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 6.

12 papers.

  1. 2025Block Rotation is All You Need for MXFP4 Quantizationcited by 20
  2. 2025ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transformscited by 10
  3. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 7
  4. 2025SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weightscited by 6
  5. 2025Beyond Outliers: A Study of Optimizers Under Quantizationcited by 6
  6. 2025Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensationcited by 3
  7. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 1
  8. 2026When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Siliconcited by 0
  9. 2026When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisationcited by 0
  10. 2026HBQ: Hierarchical Scaling Block Quantization with Hardware-Efficiency-Aware Design for Accurate LLM Inferencecited by 0
  11. 2026Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantizationcited by 0
  12. 2026ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
  2. 2025Precision Where It Matters: A Novel Spike Aware Mixed-Precision Quantization Strategy for LLaMA-based Language Models
  3. 2025SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs
  4. 2024Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
  5. 2024Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs
  6. 2025Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models
  7. 2025Turning LLM Activations Quantization-Friendly
  8. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Models