# An Empirical Study of Qwen3 Quantization

arXiv: https://arxiv.org/abs/2505.02214 · HTML: https://mlsanity.com/paper/2505.02214/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 19 references that are in the map.
- First posted on arXiv 4 May 2025.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [BiLLM: Pushing the Limit of Post-Training Quantization for LLMs](https://mlsanity.com/paper/2402.04291/) (2024)
- [RPTQ: Reorder-based Post-training Quantization for Large Language Models](https://mlsanity.com/paper/2304.01089/) (2023)
- [An empirical study of LLaMA3 quantization: from LLMs to MLLMs](https://mlsanity.com/paper/2404.14047/) (2024)
- [A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms](https://mlsanity.com/paper/2409.16694/) (2024)

## Built on by (most cited first, 21 of 21)

- [Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operations](https://mlsanity.com/paper/2505.21318/) (2025)
- [Get Experience from Practice: LLM Agents with Record & Replay](https://mlsanity.com/paper/2505.17716/) (2025)
- [Scaling Law for Quantization-Aware Training](https://mlsanity.com/paper/2505.14302/) (2025)
- [Scaling LLM Test-Time Compute with Mobile NPU on Smartphones](https://mlsanity.com/paper/2509.23324/) (2025)
- [Does quantization affect models' performance on long-context tasks?](https://mlsanity.com/paper/2505.20276/) (2025)
- [Lunguage: A Benchmark for Structured and Sequential Chest X-ray Interpretation](https://mlsanity.com/paper/2505.21190/) (2025)
- [Which Quantization Should I Use? A Unified Evaluation of llama.cpp Quantization on Llama-3.1-8B-Instruct](https://mlsanity.com/paper/2601.14277/) (2026)
- [From Shots to Stories: LLM-Assisted Video Editing with Unified Language Representations](https://mlsanity.com/paper/2505.12237/) (2025)
- [SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights](https://mlsanity.com/paper/2509.22944/) (2025)
- [ETS: Energy-Guided Test-Time Scaling for Training-Free RL Alignment](https://mlsanity.com/paper/2601.21484/) (2026)
- [SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controlling](https://mlsanity.com/paper/2508.09105/) (2025)
- [Don't be so Stief! Learning KV Cache low-rank approximation over the Stiefel manifold](https://mlsanity.com/paper/2601.21686/) (2026)
- [ASTREA: Introducing Agentic Intelligence for Orbital Thermal Autonomy](https://mlsanity.com/paper/2509.13380/) (2025)
- [Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks](https://mlsanity.com/paper/2605.01417/) (2026)
- [Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models](https://mlsanity.com/paper/2508.06974/) (2025)
- [XLQA: A Benchmark for Locale-Aware Multilingual Open-Domain Question Answering](https://mlsanity.com/paper/2508.16139/) (2025)
- [LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss](https://mlsanity.com/paper/2605.08755/) (2026)
- [Wrong Before Right: Late Rescue and Interface Failure in Aligned Language Models](https://mlsanity.com/paper/2607.04640/) (2026)
- [LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy](https://mlsanity.com/paper/2605.04295/) (2026)
- [FashionKG-RAG: Knowledge Graph-Enhanced Retrieval-Augmented Generation for Fashion Question Answering](https://mlsanity.com/paper/2608.22688/) (2026)
- [HyQuant: Hybrid-Precision Quantization for LLM Attention](https://mlsanity.com/paper/2608.27875/) (2026)

## Abstract (arXiv)

The Qwen series has emerged as a leading family of open-source Large Language Models (LLMs), demonstrating remarkable capabilities in natural language understanding tasks. With the recent release of Qwen3, which exhibits superior performance across diverse benchmarks, there is growing interest in deploying these models efficiently in resource-constrained environments. Low-bit quantization presents a promising solution, yet its impact on Qwen3's performance remains underexplored. This study conducts a systematic evaluation of Qwen3's robustness under various quantization settings, aiming to uncover both opportunities and challenges in compressing this state-of-the-art model. We rigorously assess 5 existing classic post-training quantization techniques applied to Qwen3, spanning bit-widths from 1 to 8 bits, and evaluate their effectiveness across multiple datasets. Our findings reveal that while Qwen3 maintains competitive performance at moderate bit-widths, it experiences notable degradation in linguistic tasks under ultra-low precision, underscoring the persistent hurdles in LLM compression. These results emphasize the need for further research to mitigate performance loss in extreme quantization scenarios. We anticipate that this empirical analysis will provide actionable insights for advancing quantization methods tailored to Qwen3 and future LLMs, ultimately enhancing their practicality without compromising accuracy. Our project is released on https://github.com/Efficient-ML/Qwen3-Quantization and https://huggingface.co/collections/Efficient-ML/qwen3-quantization-68164450decb1c868788cb2b.