A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPU
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Post-Training Quantization (PTQ) is crucial for efficient model deployment, yet its effectiveness on Ascend NPU remains under-explored compared to GPU architectures. This paper presents a case study of representative PTQ baselines applied to reasoning-oriented models such as DeepSeek-R1-Distill-Qwen series (1.5B/7B/14B) and QwQ-32B. We evaluate four distinct algorithms, including AWQ, GPTQ, SmoothQuant, and FlatQuant, to cover the spectrum from weight-only compression to advanced rotation-based methods. Our empirical results reveal significant platform sensitivity. While 4-bit weight-only quantization proves viable for larger models, aggressive 4-bit weight-activation schemes suffer from layer-wise calibration instability on the NPU, leading to logic collapse in long-context reasoning tasks. Conversely, standard 8-bit quantization remains numerically stable. Furthermore, a real-world INT8 deployment demonstrates that although optimized kernels reduce latency, dynamic quantization overheads currently limit end-to-end acceleration. These findings offer a practical reference for the feasibility and limitations of deploying quantized reasoning models on Ascend NPU.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 37 of its 52 references that are in the map. First posted on arXiv 6 Feb 2026.
Builds on
37 of 52 references.
Closest ancestors · 3
Same direction, newest first
- 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference13 Nov
- 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models7 Apr
- 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning6 Jan
Other references · 34
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2025Qwen3 Technical Reportcited by 8,195
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2023Let's Verify Step by Stepcited by 3,598
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
22 more
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
- 2024Qwen2.5-Coder Technical Reportcited by 1,151
- 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
- 2024SpinQuant: LLM quantization with learned rotationscited by 446
- 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
- 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
- 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
- 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
- 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
- 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
- 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
- 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
- 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 94
- 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 86
- 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
- 2024MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Modelscited by 48
- 2025Serving Large Language Models on Huawei CloudMatrix384cited by 22
- 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
- 2025DartQuant: Efficient Rotational Distribution Calibration for LLM Quantizationcited by 14
- 2025BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Modelscited by 3
- 2026W4A16 Mixed-Precision Matrix Multiplication on Decoupled Architecture: Kernel Design and Memory Bottleneck Analysis for Ascend NPUscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
- 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
- 2026Attend to Your Own Thoughts: Breaking the Barrier for Post-Training Quantization of Reasoning LLMs through the Lens of 1.58-Bit Quantization
- 2025FlexQ: Efficient Post-training INT6 Quantization for LLM Serving via Algorithm-System Co-Design
- 2025Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning
- 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
- 2026LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss
- 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving