# OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting

arXiv: https://arxiv.org/abs/2501.13987 · HTML: https://mlsanity.com/paper/2501.13987/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 23 Jan 2025.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers](https://mlsanity.com/paper/2206.01861/) (2022)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models](https://mlsanity.com/paper/2308.13137/) (2023)
- [QuIP: 2-Bit Quantization of Large Language Models With Guarantees](https://mlsanity.com/paper/2307.13304/) (2023)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving](https://mlsanity.com/paper/2310.01957/) (2023)
- [Riemannian Adaptive Optimization Methods](https://mlsanity.com/paper/1810.00760/) (2018)
- [Geoopt: Riemannian Optimization in PyTorch](https://mlsanity.com/paper/2005.02819/) (2020)
- [Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform](https://mlsanity.com/paper/2002.01113/) (2020)
- [AffineQuant: Affine Transformation Quantization for Large Language Models](https://mlsanity.com/paper/2403.12544/) (2024)
- [A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset](https://mlsanity.com/paper/1806.00358/) (2018)
- [I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models](https://mlsanity.com/paper/2405.17849/) (2024)

## Built on by (most cited first, 30 of 97)

- [MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization](https://mlsanity.com/paper/2502.00425/) (2025)
- [QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Models](https://mlsanity.com/paper/2602.20309/) (2026)
- [Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization](https://mlsanity.com/paper/2505.11695/) (2025)
- [NestQuant: Nested Lattice Quantization for Matrix Products and LLMs](https://mlsanity.com/paper/2502.09720/) (2025)
- [Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs](https://mlsanity.com/paper/2508.14896/) (2025)
- [GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning](https://mlsanity.com/paper/2502.12913/) (2025)
- [EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Models](https://mlsanity.com/paper/2508.01625/) (2025)
- [DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization](https://mlsanity.com/paper/2511.04063/) (2025)
- [FPTQuant: Function-Preserving Transforms for LLM Quantization](https://mlsanity.com/paper/2506.04985/) (2025)
- [Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models](https://mlsanity.com/paper/2603.21389/) (2026)
- [KurTail : Kurtosis-based LLM Quantization](https://mlsanity.com/paper/2503.01483/) (2025)
- [SAES-SVD: Self-Adaptive Suppression of Accumulated and Local Errors for SVD-based LLM Compression](https://mlsanity.com/paper/2602.03051/) (2026)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [Pushing the Limits of Block Rotations in Post-Training Quantization](https://mlsanity.com/paper/2601.22347/) (2026)
- [EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization](https://mlsanity.com/paper/2506.13329/) (2025)
- [SpecQuant: Spectral Decomposition and Adaptive Truncation for Ultra-Low-Bit LLMs Quantization](https://mlsanity.com/paper/2511.11663/) (2025)
- [CompTrack: Information Bottleneck-Guided Low-Rank Dynamic Token Compression for Point Cloud Tracking](https://mlsanity.com/paper/2511.15580/) (2025)
- [SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models](https://mlsanity.com/paper/2508.03351/) (2025)
- [Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment](https://mlsanity.com/paper/2509.20214/) (2025)
- [KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models](https://mlsanity.com/paper/2602.11184/) (2026)
- [NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache](https://mlsanity.com/paper/2505.18231/) (2025)
- [PCDVQ: Enhancing Vector Quantization for Large Language Models via Polar Coordinate Decoupling](https://mlsanity.com/paper/2506.05432/) (2025)
- [FocusTrack: One-Stage Focus-and-Suppress Framework for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2602.24133/) (2026)
- [DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration](https://mlsanity.com/paper/2505.23049/) (2025)
- [Optimal Brain Restoration for Joint Quantization and Sparsification of LLMs](https://mlsanity.com/paper/2509.11177/) (2025)
- [LoopQ: Quantization for Looped Language Models](https://mlsanity.com/paper/2605.16343/) (2026)
- [OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers](https://mlsanity.com/paper/2607.02461/) (2026)
- [DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization](https://mlsanity.com/paper/2608.27513/) (2026)
- [BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models](https://mlsanity.com/paper/2506.15689/) (2025)
- [VAEVQ: Enhancing Discrete Visual Tokenization through Variational Modeling](https://mlsanity.com/paper/2511.06863/) (2025)

## Abstract (arXiv)

Post-training quantization (PTQ) has emerged as a widely adopted technique for compressing and accelerating Large Language Models (LLMs). The major challenge in LLM quantization is that uneven and heavy-tailed data distributions can expand the quantization range, thereby reducing bit precision for most values. Recent methods attempt to eliminate outliers and balance inter-channel differences by employing linear transformations; however, they remain heuristic and are often overlook optimizing the data distribution across the entire quantization space.In this paper, we introduce Quantization Space Utilization Rate (QSUR), a novel metric that effectively assesses the quantizability of transformed data by measuring the space utilization of the data in the quantization space. We complement QSUR with mathematical derivations that examine the effects and limitations of various transformations, guiding our development of Orthogonal and Scaling Transformation-based Quantization (OSTQuant). OSQuant employs a learnable equivalent transformation, consisting of an orthogonal transformation and a scaling transformation, to optimize the distributions of weights and activations across the entire quantization space. Futhermore, we propose the KL-Top loss function, designed to mitigate noise during optimization while retaining richer semantic information within the limited calibration data imposed by PTQ. OSTQuant outperforms existing work on various LLMs and benchmarks. In the W4-only setting, it retains 99.5% of the floating-point accuracy. In the more challenging W4A4KV4 configuration, OSTQuant reduces the performance gap by 32% on the LLaMA-3-8B model compared to state-of-the-art methods. hrefhttps://github.com/BrotherHappy/OSTQuanthttps://github.com/BrotherHappy/OSTQuant.