mlsanityData to 7 Oct 2026

Unlocking the Unsolvable: Teacher-Guided Curriculum for Data-Efficient RLVR

arXiv 2609.13997 ↗First posted 12 Sep 2026cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has shown remarkable success in improving the mathematical reasoning of large language models. Yet problems beyond the model's current capability, where rollouts uniformly fail and no learning signal is produced, are structurally wasted despite marking the most informative training frontier. We show that these otherwise-inert problems can be unlocked via teacher-guided curriculum learning: partial reasoning traces from a stronger model create a graded difficulty landscape, and a backward-chaining curriculum progressively withdraws guidance until the model solves problems unaided. Training on only 128 unsolvable problems matches or exceeds GRPO trained on a full 2,000-problem corpus (~16x data efficiency) on the nine-benchmark average for both base models, while substantially expanding the reasoning boundary measured by pass@k at large k. Furthermore, we identify a distribution-shift cost that is particularly acute in the unsolvable-only regime and propose Monotone Frontier Curriculum (MFC), a method that monotonically drives training toward unguided solving, consistently outperforming existing curriculum methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 51 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

34 of 51 references.

Closest ancestors · 19

Same direction, newest first

  1. 2025Data-Efficient RLVR via Off-Policy Influence Guidance30 Oct
  2. 2025The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic View5 Oct
  3. 2025No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping26 Sep
  4. 2025Staying in the Sweet Spot: Responsive Reasoning Evolution via Capability-Adaptive Hint Scaffolding8 Sep
  5. 2025Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward1 Sep
  6. 2025Hard Examples Are All You Need: Maximizing GRPO Post-Training Under Annotation Budgets15 Aug
  7. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning10 Jun
  8. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  12. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search6 Jun
7 more
  1. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervision5 Jun
  2. 2024Teaching Large Language Models to Reason with Reinforcement Learning7 Mar
  3. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning8 Feb
  4. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations14 Dec
  5. 2023Let's Verify Step by Step31 May
  6. 2022Solving math word problems with process- and outcome-based feedback25 Nov
  7. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 15

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  8. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  10. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  11. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
  12. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
3 more
  1. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systemscited by 162
  2. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
  3. 2025RL for Reasoning by Adaptively Revealing Rationalescited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model
  2. 2025Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math
  3. 2026Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
  4. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning
  5. 2025Self-Evolving Curriculum for LLM Reasoning
  6. 2026On the Emergence of Implicit Curriculum in RLVR Learning Dynamics
  7. 2026DeReason: A Difficulty-Aware Curriculum Improves Decoupled SFT-then-RL Training for General Reasoning
  8. 2026VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reduction