mlsanityData to 14 Oct 2026

Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantization

arXiv 2606.09927 ↗First posted 7 Jun 2026cs.LG cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) is one of the most practical ways to reduce the serving cost of Large Language Models (LLMs), but activation quantization remains difficult because outlier-dominated channels lead to large quantization errors. This paper investigates whether part of this degradation is caused by over-migration in scaling-based equivalent transformations. We introduce a quantile-robust scaling policy for SmoothRot-style transforms by replacing max-based activation statistics with high quantiles, and we complement it with constrained gradient-based optimization of channel scales. On LLaMA-3.2-1B under W4A4 quantization, quantile-only policy search improves selected-layer error by 11.1% over the SmoothRot baseline, joint (alpha, q) search improves it by 12%, and training reaches 18.5%. Replaying the best selected-layer policy on all decoder-block down-projection layers reduces the corresponding full-layer mean error from 97.51 to 78.08 (19.9%). The results show that robust migration control and lightweight scale learning provide consistent gains over max-based fixed policies while preserving the equivalent-transform framework.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 7 Jun 2026.

Builds on

15 of 18 references.

References · 15

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  7. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  8. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  9. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  10. 2024Massive Activations in Large Language Modelscited by 294
  11. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  12. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
3 more
  1. 2024Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMscited by 21
  2. 2025SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMscited by 4
  3. 2025Turning LLM Activations Quantization-Friendlycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Adaptive Layer-Wise Transformations for Post-Training Quantization of Large Language Models
  2. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
  3. 2024DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotation
  4. 2026Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals
  5. 2026Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs
  6. 2026InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
  7. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
  8. 2025Outlier Smoothing with Closed-Form Rotations for W4A4 Large Language Model Quantization