mlsanityData to 7 Oct 2026

Efficient Reinforcement Finetuning via Adaptive Curriculum Learning

arXiv 2504.05520 ↗First posted 7 Apr 2025cs.LG cs.CLcited by 88

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement finetuning (RFT) has shown great potential for enhancing the mathematical reasoning capabilities of large language models (LLMs), but it is often sample- and compute-inefficient, requiring extensive training. In this work, we introduce AdaRFT (Adaptive Curriculum Reinforcement Finetuning), a method that significantly improves the efficiency of RFT through adaptive curriculum learning. AdaRFT dynamically adjusts the difficulty of training problems based on the model's recent reward signals, ensuring that the model consistently trains on tasks that are challenging but solvable. This adaptive sampling strategy accelerates learning by maintaining an optimal difficulty range, avoiding wasted computation on problems that are too easy or too hard. AdaRFT requires only a lightweight extension to standard RFT algorithms like Proximal Policy Optimization (PPO), without modifying the reward function or model architecture. Experiments on competition-level math datasets demonstrate that AdaRFT improves convergence efficiency and reasoning performance. Given problem-level difficulty annotations, AdaRFT reduces RFT training time by up to 2 times across data distributions and model scales, offering a more scalable and effective RFT framework.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 54 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

36 of 54 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning4 Apr
  2. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  4. 2025LIMR: Less is More for RL Scaling17 Feb
  5. 2025Process Reinforcement through Implicit Rewards3 Feb
  6. 2023Let's Verify Step by Step31 May
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  8. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment13 Apr
  9. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 27

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  4. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  5. 2025s1: Simple test-time scalingcited by 1,311
  6. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  7. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  8. 2025LIMO: Less is More for Reasoningcited by 488
  9. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  10. 2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
  11. 2017Teacher-Student Curriculum Learningcited by 197
  12. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 186
15 more
  1. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 184
  2. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  3. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  4. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 172
  5. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  6. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  7. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 113
  8. 2014Learning to Executecited by 105
  9. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 80
  10. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  11. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Modelscited by 78
  12. 2024Offline Regularised Reinforcement Learning for Large Language Models Alignmentcited by 48
  13. 2019Unsupervised Curricula for Visual Meta-Reinforcement Learningcited by 33
  14. 2018Mix&Match - Agent Curricula for Reinforcement Learningcited by 26
  15. 2025Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedbackcited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 47; 2026: 41.

88 papers.

Sort:

By area:

  1. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  2. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  3. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
  4. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  5. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 68
  6. 2025UFT: Unifying Supervised and Reinforcement Fine-Tuningcited by 58
  7. 2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 56
  8. 2025Do Not Let Low-Probability Tokens Over-Dominate in RL for LLMscited by 43
  9. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  10. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 40
  11. 2025RL Tango: Reinforcing Generator and Verifier Together for Language Reasoningcited by 39
  12. 2025SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoningcited by 39
38 more
  1. 2026TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agentscited by 38
  2. 2025The Hallucination Tax of Reinforcement Finetuningcited by 35
  3. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 33
  4. 2025Curriculum-RLAIF: Curriculum Alignment with Reinforcement Learning from AI Feedbackcited by 30
  5. 2025SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learningcited by 27
  6. 2025Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RLcited by 23
  7. 2025Reinforce-Ada: An Adaptive Sampling Framework under Non-linear RL Objectivescited by 23
  8. 2026Experiential Reinforcement Learningcited by 21
  9. 2025Learning Like Humans: Advancing LLM Reasoning Capabilities via Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulationcited by 20
  10. 2025Imbalanced Gradients in RL Post-Training of Multi-Task LLMscited by 19
  11. 2025CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMscited by 16
  12. 2025RL for Reasoning by Adaptively Revealing Rationalescited by 15
  13. 2025Angles Don't Lie: Unlocking Training-Efficient RL Through the Model's Own Signalscited by 13
  14. 2025Aware First, Think Less: Dynamic Boundary Self-Awareness Drives Extreme Reasoning Efficiency in Large Language Modelscited by 13
  15. 2025SWE-Bench-CL: Continual Learning for Coding Agentscited by 12
  16. 2025A Snapshot of Influence: A Local Data Attribution Framework for Online Reinforcement Learningcited by 11
  17. 2025CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoningcited by 11
  18. 2026CoBA-RL: Capability-Oriented Budget Allocation for Reinforcement Learning in LLMscited by 11
  19. 2025G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidancecited by 10
  20. 2026Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoningcited by 10
  21. 2026Train at Moving Edge: Online-Verified Prompt Selection for Efficient RL Training of Large Reasoning Modelcited by 10
  22. 2025Writing-RL: Advancing Long-form Writing via Adaptive Curriculum Reinforcement Learningcited by 9
  23. 2026VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reductioncited by 9
  24. 2025C2GSPG: Confidence-calibrated Group Sequence Policy Gradient towards Self-aware Reasoningcited by 8
  25. 2025h1: Bootstrapping LLMs to Reason over Longer Horizons via Reinforcement Learningcited by 8
  26. 2025Towards Understanding Self-play for LLM Reasoningcited by 8
  27. 2025EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learningcited by 7
  28. 2026SCALER:Synthetic Scalable Adaptive Learning Environment for Reasoningcited by 7
  29. 2026Prompt replay: speeding up grpo with on-policy reuse of high-signal promptscited by 7
  30. 2025RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Samplingcited by 6
  31. 2026Self-Distilled Policy Gradientcited by 6
  32. 2025M3OOD: Automatic Selection of Multimodal OOD Detectorscited by 5
  33. 2026RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoningcited by 5
  34. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  35. 2025How Many Code and Test Cases Are Enough? Evaluating Test Cases Generation from a Binary-Matrix Perspectivecited by 4
  36. 2025Provable Benefit of Curriculum in Transformer Tree-Reasoning Post-Trainingcited by 4
  37. 2026HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learningcited by 4
  38. 2026SortedRL: Accelerating RL Training for LLMs through Online Length-Aware Schedulingcited by 4

50 most cited of 88 All 88 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting
  2. 2025Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMs
  3. 2025Progressive Mastery: Customized Curriculum Learning with Guided Prompting for Mathematical Reasoning
  4. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models
  5. 2025Boosting Accuracy and Efficiency of Budget Forcing in LLMs via Reinforcement Learning for Mathematical Reasoning
  6. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification
  7. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning
  8. 2026IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoning