mlsanityData to 14 Oct 2026

SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs

arXiv 2506.05413 ↗First posted 4 Jun 2025cs.CL cs.AI cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We present SmoothRot, a novel post-training quantization technique to enhance the efficiency of 4-bit quantization in Large Language Models (LLMs). SmoothRot addresses the critical challenge of massive activation outliers, by integrating channel-wise scaling with Hadamard transformations. Our technique effectively transforms extreme outliers into quantization-friendly activations, significantly improving quantization accuracy. Experiments conducted on popular LLMs (LLaMA2 7B, LLaMA3.1 8B, and Mistral 7B) demonstrate that SmoothRot consistently reduces the performance gap between quantized and FP16 models by approximately 10-30% across language generation and zero-shot reasoning tasks, without introducing additional inference latency. Code is available at https://github.com/czakop/smoothrot.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

26 of 34 references.

References · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2023Mistral 7Bcited by 3,164
  7. 2016Pointer Sentinel Mixture Modelscited by 3,096
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  11. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  12. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
14 more
  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  2. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  3. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  4. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  5. 2024Massive Activations in Large Language Modelscited by 294
  6. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 204
  7. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  8. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  9. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  10. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  11. 2024Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantizationcited by 33
  12. 2024Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMscited by 21
  13. 2024DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotationcited by 17
  14. 2025Turning LLM Activations Quantization-Friendlycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantizationcited by 0
  2. 2026Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residualscited by 0
  3. 2026FoldQuantVLA: Native Low-Bit Quantization of Vision-Language-Action Models via Consistent Foldingcited by 0
  4. 2026DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM
  2. 2025Precision Where It Matters: A Novel Spike Aware Mixed-Precision Quantization Strategy for LLaMA-based Language Models
  3. 2024OutlierTune: Efficient Channel-Wise Quantization for Large Language Models
  4. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
  5. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
  6. 2025FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization
  7. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsetting
  8. 2026Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs