mlsanityData to 14 Oct 2026

Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models

arXiv 2504.04823 ↗First posted 7 Apr 2025cs.CL cs.AIcited by 73

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent advancements in reasoning language models have demonstrated remarkable performance in complex tasks, but their extended chain-of-thought reasoning process increases inference overhead. While quantization has been widely adopted to reduce the inference cost of large language models, its impact on reasoning models remains understudied. In this paper, we conduct the first systematic study on quantized reasoning models, evaluating the open-sourced DeepSeek-R1-Distilled Qwen and LLaMA families ranging from 1.5B to 70B parameters, QwQ-32B, and Qwen3-8B. Our investigation covers weight, KV cache, and activation quantization using state-of-the-art algorithms at varying bit-widths, with extensive evaluation across mathematical (AIME, MATH-500), scientific (GPQA), and programming (LiveCodeBench) reasoning benchmarks. Our findings reveal that while lossless quantization can be achieved with W8A8 or W4A16 quantization, lower bit-widths introduce significant accuracy risks. We further identify model size, model origin, and task difficulty as critical determinants of performance. Contrary to expectations, quantized models do not exhibit increased output lengths. In addition, strategically scaling the model sizes or reasoning steps can effectively enhance the performance. All quantized models and codes are open-sourced in https://github.com/ruikangliu/Quantized-Reasoning-Models.

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 36 of its 56 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

36 of 56 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  2. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling10 Feb
  3. 2025s1: Simple test-time scaling31 Jan
  4. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  5. 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning6 Jan
  6. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  7. 2024Token-Budget-Aware LLM Reasoning24 Dec
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 28

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2023Let's Verify Step by Stepcited by 3,789
  9. 2023Qwen Technical Reportcited by 3,344
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
16 more
  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  3. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  4. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  5. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  7. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  8. 2025LIMO: Less is More for Reasoningcited by 498
  9. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  10. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  11. 2023Microscaling Data Formats for Deep Learningcited by 152
  12. 2024Compact Language Models via Pruning and Knowledge Distillationcited by 148
  13. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  14. 2024Evaluating Quantized Large Language Modelscited by 77
  15. 2023On the Impact of Calibration Data in Post-training Quantization and Pruningcited by 58
  16. 2024IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intactcited by 52

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 25; 2026: 48.

73 papers.

Sort:

By area:

  1. 2025Efficient Reasoning Models: A Surveycited by 95
  2. 2025SeerAttention-R: Sparse Attention Adaptation for Long Reasoningcited by 30
  3. 202570% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float (DFloat11)cited by 24
  4. 2025VeriThinker: Learning to Verify Makes Reasoning Model Efficientcited by 24
  5. 2025Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMscited by 23
  6. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  7. 2025Scaling Law for Quantization-Aware Trainingcited by 17
  8. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 17
  9. 2025StreamingThinker: Large Language Models Can Think While Readingcited by 15
  10. 2025Scaling LLM Test-Time Compute with Mobile NPU on Smartphonescited by 11
  11. 2025Does quantization affect models' performance on long-context tasks?cited by 10
  12. 2025Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boostcited by 10
38 more
  1. 2026Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formatscited by 10
  2. 2026What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Studycited by 10
  3. 2026HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharingcited by 10
  4. 2026Quantized Reasoning Models Think They Need to Think Longer, but They Do Notcited by 10
  5. 2025MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoningcited by 8
  6. 2026Which Quantization Should I Use? A Unified Evaluation of llama.cpp Quantization on Llama-3.1-8B-Instructcited by 8
  7. 2026M2XFP: A Metadata-Augmented Microscaling Data Format for Efficient Low-bit Quantizationcited by 8
  8. 2026ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compressioncited by 8
  9. 2025Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoningcited by 7
  10. 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimizationcited by 6
  11. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospectscited by 5
  12. 2025SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Modelscited by 5
  13. 2026QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signalscited by 5
  14. 2026From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantizationcited by 5
  15. 2025A Simple Linear Patch Revives Layer-Pruned Large Language Modelscited by 4
  16. 2025The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Mergingcited by 4
  17. 2025Not All Bits Are Equal: Scale-Dependent Memory Optimization Strategies for Reasoning Modelscited by 4
  18. 2026ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Schedulingcited by 4
  19. 2026FreeAct: Freeing Activations for LLM Quantizationcited by 4
  20. 2026Efficient Reasoning with Balanced Thinkingcited by 4
  21. 2026ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Modelscited by 3
  22. 2026Search Your Block Floating Point Scales!cited by 3
  23. 2026What Survives When You Compress a Recursive Reasoner for the Edge?cited by 3
  24. 2026KronQ: LLM Quantization via Kronecker-Factored Hessiancited by 3
  25. 2025Quantitative Analysis of Performance Drop in DeepSeek Model Quantizationcited by 2
  26. 2025Attention-Aligned Reasoning for Large Language Modelscited by 2
  27. 2025Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Modelscited by 2
  28. 2026Float8@2bits: Entropy Coding Enables Data-Free Model Compressioncited by 2
  29. 2026Dynamic Mixed-Precision Routing for Efficient Multi-step LLM Interactioncited by 2
  30. 2026The Quantization Trap: Breaking Linear Scaling Laws in Multi-Hop Reasoningcited by 2
  31. 2026Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initializationcited by 2
  32. 2026Quantizing Recursive Reasoning Modelscited by 2
  33. 2025EffiReason-Bench: A Unified Benchmark for Evaluating and Advancing Efficient Reasoning in Large Language Modelscited by 1
  34. 2025Post-Training Quantization of OpenPangu Models for Efficient Deployment on Atlas A2cited by 1
  35. 2026A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPUcited by 1
  36. 2026QuantClaw: Precision Where It Matters for OpenClawcited by 1
  37. 2026LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Losscited by 1
  38. 2026Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recoverycited by 1

50 most cited of 73 All 73 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models
  2. 2026Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
  3. 2024Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant
  4. 2026Attend to Your Own Thoughts: Breaking the Barrier for Post-Training Quantization of Reasoning LLMs through the Lens of 1.58-Bit Quantization
  5. 2025When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models
  6. 2026Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning
  7. 2025Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs
  8. 2023Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study