mlsanityData to 7 Oct 2026

Post-Training Quantization of OpenPangu Models for Efficient Deployment on Atlas A2

arXiv 2512.23367 ↗First posted 29 Dec 2025cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Huawei's openPangu-Embedded-1B and openPangu-Embedded-7B are variants of the openPangu large language model, designed for efficient deployment on Ascend NPUs. The 7B variant supports three distinct Chain-of-Thought (CoT) reasoning paradigms, namely slowthink, autothink, and nothink, while the 1B variant operates exclusively in the nothink mode, which employs condensed reasoning for higher efficiency. Although CoT reasoning enhances capability, the generation of extended reasoning traces introduces substantial memory and latency overheads, posing challenges for practical deployment on Ascend NPUs. This paper addresses these computational constraints by leveraging low-bit quantization, which transforms FP16 computations into more efficient integer arithmetic. We introduce a unified low-bit inference framework, supporting INT8 (W8A8) and W4A8 quantization, specifically optimized for openPangu-Embedded models on the Atlas A2. Our comprehensive evaluation on code generation benchmarks (HumanEval and MBPP) demonstrates the efficacy of this approach. INT8 quantization consistently preserves over 90% of the FP16 baseline accuracy and achieves a 1.5x prefill speedup on the Atlas A2. Furthermore, W4A8 quantization significantly reduces memory consumption, albeit with a moderate trade-off in accuracy. These findings collectively indicate that low-bit quantization effectively facilitates efficient CoT reasoning on Ascend NPUs, maintaining high model fidelity.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 29 Dec 2025.

Builds on

27 of 32 references.

References · 27

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  7. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  8. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  9. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  10. 2021A White Paper on Neural Network Quantizationcited by 453
  11. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  12. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
15 more
  1. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  2. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  3. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
  4. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  5. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 113
  6. 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 98
  7. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
  8. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 68
  9. 2025Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognitioncited by 61
  10. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 47
  11. 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoningcited by 36
  12. 2023Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantizationcited by 20
  13. 2025MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Modelscited by 11
  14. 2025Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniquescited by 9
  15. 2024CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compressioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoningcited by 10

Similar, not cited

Close in wording, no citation either way

  1. 2023Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
  2. 2026A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPU
  3. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency
  4. 2025BitNet b1.58 2B4T Technical Report
  5. 2026From Bits to Chips: An LLM-based Hardware-Aware Quantization Agent for Streamlined Deployment of LLMs
  6. 2025On-Device Qwen2.5: Efficient LLM Inference with Model Compression and Hardware Acceleration
  7. 2026Unified Deployment-Aware Evaluation of Open Reasoning Language Models
  8. 2025Scaling LLM Test-Time Compute with Mobile NPU on Smartphones