mlsanityData to 14 Oct 2026

Turning LLM Activations Quantization-Friendly

arXiv 2506.01967 ↗First posted 11 May 2025cs.LG cs.AI cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Quantization effectively reduces the serving costs of Large Language Models (LLMs) by speeding up data movement through compressed parameters and enabling faster operations via integer arithmetic. However, activating integer arithmetic requires quantizing both weights and activations, which poses challenges due to the significant outliers in LLMs that increase quantization error. In this work, we investigate these outliers with an emphasis on their effect on layer-wise quantization error, then examine how smoothing and rotation transform the observed values. Our primary contributions include introducing a new metric to measure and visualize quantization difficulty based on channel magnitudes, as well as proposing a hybrid approach that applies channel-wise scaling before rotation, supported by a mathematical formulation of its benefits.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 26 references that are in the map. First posted on arXiv 11 May 2025.

Builds on

21 of 26 references.

References · 21

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Mistral 7Bcited by 3,164
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  9. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  10. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  11. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  12. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
9 more
  1. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  2. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  3. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  4. 2023A Survey on Model Compression for Large Language Modelscited by 269
  5. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  6. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  7. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  8. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  9. 2024Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMscited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMscited by 4
  2. 2026Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals
  2. 2026InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
  3. 2025Rethinking the Outlier Distribution in Large Language Models: An In-depth Study
  4. 2025Why Do Some Inputs Break Low-Bit LLM Quantization?
  5. 2024Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs
  6. 2024Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels
  7. 2025Achieving binary weight and activation for LLMs using Post-Training Quantization
  8. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Models