# AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting

arXiv: https://arxiv.org/abs/2511.09478 · HTML: https://mlsanity.com/paper/2511.09478/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- First posted on arXiv 12 Nov 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)
- Deep reinforcement learning methods / [RL training of language and agent models](https://mlsanity.com/deeprl/rl-training-of-language-and-agent-models/)

## Built on by (most cited first, 20 of 20)

- [Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulation](https://mlsanity.com/paper/2601.20614/) (2026)
- [Taming Preference Mode Collapse via Directional Decoupling Alignment in Diffusion Reinforcement Learning](https://mlsanity.com/paper/2512.24146/) (2025)
- [Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models](https://mlsanity.com/paper/2601.20354/) (2026)
- [LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks](https://mlsanity.com/paper/2608.01964/) (2026)
- [ADHint: Adaptive Hints with Difficulty Priors for Reinforcement Learning](https://mlsanity.com/paper/2512.13095/) (2025)
- [CoBA-RL: Capability-Oriented Budget Allocation for Reinforcement Learning in LLMs](https://mlsanity.com/paper/2602.03048/) (2026)
- [Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization](https://mlsanity.com/paper/2601.05432/) (2026)
- [FASA: Frequency-aware Sparse Attention](https://mlsanity.com/paper/2602.03152/) (2026)
- [Off-Context GRPO: Learning to Reason on Hard Problems using Privileged Information](https://mlsanity.com/paper/2607.19313/) (2026)
- [Video-CoE: Reinforcing Video Event Prediction via Chain of Events](https://mlsanity.com/paper/2603.14935/) (2026)
- [Learning Agentic Policy from Action Guidance](https://mlsanity.com/paper/2605.12004/) (2026)
- [VideoCuRL: Video Curriculum Reinforcement Learning with Orthogonal Difficulty Decomposition](https://mlsanity.com/paper/2601.00887/) (2025)
- [Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning](https://mlsanity.com/paper/2608.03571/) (2026)
- [RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents](https://mlsanity.com/paper/2606.19047/) (2026)
- [AR-MAP: Are Autoregressive Large Language Models Implicit Teachers for Diffusion Large Language Models?](https://mlsanity.com/paper/2602.02178/) (2026)
- [D2Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learning](https://mlsanity.com/paper/2605.17037/) (2026)
- [Frontier Learning: Training LLM Reasoners at the Edge of Capability](https://mlsanity.com/paper/2609.35426/) (2026)
- [PB-GRPO: Learning Socially Adaptive LLM Agents from Persona-Driven Simulation with Preference-Batched GRPO](https://mlsanity.com/paper/2610.04132/) (2026)
- [SWEAR: Section-Wise Efficient Audio Reasoning via Self-Rated Difficulty](https://mlsanity.com/paper/2610.13081/) (2026)
- [A Step Harder, Not a Step Too Far: The Benefits and Limits of Curriculum Difficulty Progression in GRPO](https://mlsanity.com/paper/2610.13669/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) has demonstrated considerable potential for enhancing reasoning in large language models (LLMs). However, existing methods suffer from Gradient Starvation and Policy Degradation when training directly on samples with mixed difficulty. To mitigate this, prior approaches leverage Chain-of-Thought (CoT) data, but the construction of high-quality CoT annotations remains labor-intensive. Alternatively, curriculum learning strategies have been explored but frequently encounter challenges, such as difficulty mismatch, reliance on manual curriculum design, and catastrophic forgetting. To address these issues, we propose AdaCuRL, a Adaptive Curriculum Reinforcement Learning framework that integrates coarse-to-fine difficulty estimation with adaptive curriculum scheduling. This approach dynamically aligns data difficulty with model capability and incorporates a data revisitation mechanism to mitigate catastrophic forgetting. Furthermore, AdaCuRL employs adaptive reference and sparse KL strategies to prevent Policy Degradation. Extensive experiments across diverse reasoning benchmarks demonstrate that AdaCuRL consistently achieves significant performance improvements on both LLMs and MLLMs.