mlsanityData to 14 Oct 2026

QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation

arXiv 2507.13266 ↗First posted 17 Jul 2025cs.CL cs.AIcited by 44

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning (RL) has emerged as a central paradigm for training large language models (LLMs) in reasoning tasks. Yet recent studies question RL's ability to incentivize reasoning capacity beyond the base model. This raises a key challenge: how can RL be adapted to solve harder reasoning problems more effectively? To address this challenge, we propose a simple yet effective strategy via Question Augmentation: introduce partial solutions during training to reduce problem difficulty and provide more informative learning signals. Our method, QuestA, when applied during RL training on math reasoning tasks, not only improves pass@1 but also pass@k-particularly on problems where standard RL struggles to make progress. This enables continual improvement over strong open-source models such as DeepScaleR and OpenMath Nemotron, further enhancing their reasoning capabilities. We achieve new state-of-the-art results on math benchmarks using 1.5B-parameter models: 72.50% (+10.73%) on AIME24, 62.29% (+12.79%) on AIME25, and 41.67% (+10.11%) on HMMT25. Code, data and model are available at https://github.com/foreverlasting1202/QuestA.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 33 references that are in the map. First posted on arXiv 17 Jul 2025.

Builds on

26 of 33 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025StepHint: Multi-level Stepwise Hints Enhance Reinforcement Learning to Reason3 Jul
  2. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs17 Jun
  3. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  4. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning7 Jun
  5. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning2 Jun
  6. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning2 Jun
  7. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models30 May
  8. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning30 May
  9. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  10. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  11. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  12. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining10 Apr
5 more
  1. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  2. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning10 Mar
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  4. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning8 Feb
  5. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 9

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  5. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  6. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
  7. 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 264
  8. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 174
  9. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Modelscited by 54

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 32.

44 papers.

Sort:

By area:

  1. 2025Beyond Ten Turns: Unlocking Long-Horizon Agentic Search with Large-Scale Asynchronous RLcited by 150
  2. 2025JustRL: Scaling a 1.5B LLM with a Simple RL Recipecited by 64
  3. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 53
  4. 2026POPE: Learning to Reason on Hard Problems via Privileged On-Policy Explorationcited by 42
  5. 2025Nudging the Boundaries of LLM Reasoningcited by 26
  6. 2025Beyond the Sampled Token: Preserving Candidate Support in RLVRcited by 18
  7. 2026Reuse your FLOPs: Scaling RL on Hard Problems by Conditioning on Very Off-Policy Prefixescited by 16
  8. 2025Don't Tell the Answer, Truly Guide the Reasoning During RL Rolloutscited by 14
  9. 2026InT: Self-Proposed Interventions Enable Credit Assignment in LLM Reasoningcited by 14
  10. 2025EntroPIC: Towards Stable Long-Term Training of LLMs via Entropy Stabilization with Proportional-Integral Controlcited by 13
  11. 2026Learn Hard Problems During RL with Reference Guided Fine-tuningcited by 12
  12. 2025Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learningcited by 11
32 more
  1. 2026KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidancecited by 8
  2. 2025Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMscited by 7
  3. 2026Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptationcited by 7
  4. 2026Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewardscited by 6
  5. 2026Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Modelscited by 6
  6. 2026Hindsight Hint Distillation: Scaffolded Reasoning for SWE Agents from CoT-free Answerscited by 5
  7. 2025EmbodiedBrain: Expanding Performance Boundaries of Task Planning for Embodied Intelligencecited by 4
  8. 2026Cog-DRIFT: Exploration on Adaptively Reformulated Instances Enables Learning from Hard Reasoning Problemscited by 4
  9. 2025IntAttention: A Fully Integer Attention Pipeline for Efficient Edge Inferencecited by 3
  10. 2026P^2O: Joint Policy and Prompt Optimizationcited by 3
  11. 2026LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidancecited by 3
  12. 2026Breaking the Solver Bottleneck: Training Task Generators at the Learnable Frontiercited by 3
  13. 2025RLoop: An Self-Improving Framework for Reinforcement Learning with Iterative Policy Initializationcited by 2
  14. 2025PIGEON: VLM-Driven Object Navigation via Points of Interest Selectioncited by 1
  15. 2026PrAg-PO: Prompt Augmented Policy Optimization for Robust and Diverse Mathematical Reasoningcited by 1
  16. 2026Asking the Right Questions: Improving Reasoning with Generated Stepping Stonescited by 1
  17. 2026Mechanistically Interpreting the Role of Sample Difficulty in RLVR for LLMscited by 1
  18. 2026Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuningcited by 1
  19. 2026D2Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learningcited by 0
  20. 2026Beyond Execution: Static-Analysis Rewards and Hint-Conditioned Diffusion RL for Code Generationcited by 0
  21. 2026TD-Grokking: Learning from Zero-Reward Problems by Training-Time Decompositioncited by 0
  22. 2026The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithmscited by 0
  23. 2026AdaStop: Cost-Aware Early Stopping for DNN Test Selectioncited by 0
  24. 2026AdaKP: Online Adaptive Knowledge-Point Selection for Reasoning-Oriented Reinforcement Learningcited by 0
  25. 2026Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Taskscited by 0
  26. 2026Teach to Learn: Hint Annealing for Self-improving LLM Reasoningcited by 0
  27. 2026Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVRcited by 0
  28. 2026Understanding Enrichment in Reinforcement Learningcited by 0
  29. 2026E2-OPSD: Taming Entropy Overshoot in On-Policy Self-Distillationcited by 0
  30. 2026Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffoldingcited by 0
  31. 2026MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesiscited by 0
  32. 2026MENTOR: Learning from Failure with On-Policy Pedagogical Interventioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  3. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
  4. 2025How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study
  5. 2025General-Reasoner: Advancing LLM Reasoning Across All Domains
  6. 2025Training Language Models to Reason Efficiently
  7. 2025Learning to Reason at the Frontier of Learnability
  8. 2025Thinker: Learning to Think Fast and Slow