mlsanityData to 14 Oct 2026

The Impact of Quantization on Large Reasoning Model Reinforcement Learning

arXiv 2511.15694 ↗First posted 19 Nov 2025cs.LGcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Strong reasoning capabilities can now be achieved by large-scale reinforcement learning (RL) without any supervised fine-tuning. Although post-training quantization (PTQ) and quantization-aware training (QAT) are well studied in the context of fine-tuning, how quantization impacts RL in large reasoning models (LRMs) remains an open question. To answer this question, we conducted systematic experiments and discovered a significant gap in reasoning performance on mathematical benchmarks between post-RL quantized models and their quantization-aware RL optimized counterparts. Our findings suggest that quantization-aware RL training negatively impacted the learning process, whereas PTQ and QLoRA led to greater performance.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 12 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

10 of 12 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar

Other references · 9

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  7. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  8. 2019QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learningcited by 19
  9. 2024Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Losscited by 1
  2. 2026BitRL: Reinforcement Learning with 1-bit Quantized Language Models for Resource-Constrained Edge Deploymentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
  2. 2026QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals
  3. 2025QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs
  4. 2025Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
  5. 2026Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
  6. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  7. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
  8. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models