mlsanityData to 14 Oct 2026

ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference

arXiv 2511.10645 ↗First posted 13 Nov 2025cs.CLcited by 17

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Post-training quantization (PTQ) compresses the weights and activations of large language models (LLMs) into low-precision representations to reduce memory footprint and accelerate inference. However, the presence of outliers in weights and activations often leads to large quantization errors and severe accuracy degradation, especially in recent reasoning LLMs where errors accumulate across long chains of thought. Existing PTQ methods either fail to sufficiently suppress outliers or introduce significant overhead during inference. In this paper, we propose Pairwise Rotation Quantization (ParoQuant), a PTQ method that combines hardware-efficient and optimizable independent Givens rotations with channel-wise scaling to even out the magnitudes across channels and narrow the dynamic range within each quantization group, effectively addressing the outlier issue. We further co-design the inference kernel to fully exploit GPU parallelism and keep the rotations and scaling lightweight at runtime. Under weight-only quantization, ParoQuant achieves an average 2.4% accuracy improvement over AWQ on reasoning tasks, with less than 10% overhead. ParoQuant also matches the accuracy of state-of-the-art weight-activation quantization methods. This paves the way for more efficient and accurate deployment of reasoning LLMs.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 36 of its 45 references that are in the map. First posted on arXiv 13 Nov 2025.

Builds on

36 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models7 Apr

Other references · 35

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  11. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
23 more
  1. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  5. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  7. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  8. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  9. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  10. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  11. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  12. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  13. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  14. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 272
  15. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  16. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 182
  17. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  18. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  19. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  20. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  21. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  22. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 32
  23. 2024QSpec: Speculative Decoding with Complementary Quantization Schemescited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 17.

17 papers.

Sort:

By area:

  1. 2026Adaptive Block-Scaled Data Typescited by 4
  2. 2026ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximationcited by 2
  3. 2026Saliency-Aware Regularized Quantization Calibration for Large Language Modelscited by 2
  4. 2026Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMscited by 2
  5. 2026Prefix Sliding for efficient test-time scalingcited by 2
  6. 2026A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPUcited by 1
  7. 2026LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Losscited by 1
  8. 2026Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floorcited by 1
  9. 2026Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Modelscited by 1
  10. 2026GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetriescited by 1
  11. 2026The Asymmetric Harms of LLM Compressioncited by 1
  12. 2026SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Modelscited by 1
5 more
  1. 2026Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Datacited by 0
  2. 2026FlashDrive: Flash Vision-Language-Action Inference for Autonomous Drivingcited by 0
  3. 2026When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisationcited by 0
  4. 2026The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantizationcited by 0
  5. 2026SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM
  2. 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
  3. 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
  4. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
  5. 2024LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inference
  6. 2026RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models
  7. 2025SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs
  8. 2025Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning