# ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs arXiv: https://arxiv.org/abs/2609.36120 · HTML: https://mlsanity.com/paper/2609.36120/ ## Key facts - No citing papers in the map yet (data through 14 Oct 2026). - Builds on 18 of its 33 references that are in the map. - First posted on arXiv 28 Sep 2026. ## Builds on (most cited first, 18 of 18) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018) - [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018) - [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019) - [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019) - [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019) - [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018) - [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022) - [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024) - [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024) - [OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models](https://mlsanity.com/paper/2308.13137/) (2023) - [QuIP: 2-Bit Quantization of Large Language Models With Guarantees](https://mlsanity.com/paper/2307.13304/) (2023) - [OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting](https://mlsanity.com/paper/2501.13987/) (2025) - [Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling](https://mlsanity.com/paper/2304.09145/) (2023) - [GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration](https://mlsanity.com/paper/2504.02692/) (2025) - [DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization](https://mlsanity.com/paper/2511.04063/) (2025) - [KurTail : Kurtosis-based LLM Quantization](https://mlsanity.com/paper/2503.01483/) (2025) - [ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models](https://mlsanity.com/paper/2605.11222/) (2026) ## Abstract (arXiv) Learned rotations play an important role in enabling low-bit weight and activation quantization of large language models by smoothing outliers in the activation distribution. State-of-the-art approaches include gradient-based procedures such as SpinQuant and computationally friendlier gradient-free approaches such as DartQuant, but both remain hard to scale to the largest architectures. To address the computational bottlenecks in gradient-free rotation learning, we introduce two ideas for efficiency, (i) a data selection procedure which reduces the required number of calibration data points, and (ii) an exact reduction of the associated optimization on this reduced calibration set. Our data selection procedure exploits the geometric structure of the convex hull of the activations. Using this idea, we show that a carefully selected calibration set with several orders of magnitude fewer activations than state-of-the-art rotation-based methods can match their performance in low-bit quantization settings. Under this extreme data efficiency, the selected activations span an r-dimensional subspace with r