mlsanityData to 14 Oct 2026

R-Zero: Self-Evolving Reasoning LLM from Zero Data

arXiv 2508.05004 ↗First posted 7 Aug 2025cs.LG cs.AI cs.CLcited by 228

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Self-evolving Large Language Models (LLMs) offer a scalable path toward super-intelligence by autonomously generating, refining, and learning from their own experiences. However, existing methods for training such models still rely heavily on vast human-curated tasks and labels, typically via fine-tuning or reinforcement learning, which poses a fundamental bottleneck to advancing AI systems toward capabilities beyond human intelligence. To overcome this limitation, we introduce R-Zero, a fully autonomous framework that generates its own training data from scratch. Starting from a single base LLM, R-Zero initializes two independent models with distinct roles, a Challenger and a Solver. These models are optimized separately and co-evolve through interaction: the Challenger is rewarded for proposing tasks near the edge of the Solver capability, and the Solver is rewarded for solving increasingly challenging tasks posed by the Challenger. This process yields a targeted, self-improving curriculum without any pre-existing tasks and labels. Empirically, R-Zero substantially improves reasoning capability across different backbone LLMs, e.g., boosting the Qwen3-4B-Base by +6.49 on math-reasoning benchmarks and +7.54 on general-domain reasoning benchmarks.

Key facts. Cited by 228 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 57 of its 69 references that are in the map. First posted on arXiv 7 Aug 2025.

Builds on

57 of 69 references.

References · 57

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  9. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  10. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  11. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  12. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
38 more
  1. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  2. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  3. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  4. 2025Reasoning with Exploration: An Entropy Perspectivecited by 281
  5. 2025TTRL: Test-Time Reinforcement Learningcited by 275
  6. 2025Learning to Reason without External Rewardscited by 210
  7. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
  8. 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 205
  9. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 166
  10. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 162
  11. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 158
  12. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 156
  13. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 135
  14. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
  15. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivizationcited by 128
  16. 2025BIG-Bench Extra Hardcited by 125
  17. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scalingcited by 113
  18. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 106
  19. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 104
  20. 2025Maximizing Confidence Alone Improves Reasoningcited by 100
  21. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 93
  22. 2024A Survey on Self-Evolution of Large Language Modelscited by 91
  23. 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Modelscited by 82
  24. 2023Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loopcited by 79
  25. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 79
  26. 2025Efficient Test-Time Scaling via Self-Calibrationcited by 75
  27. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learningcited by 69
  28. 2024Strong Model Collapsecited by 65
  29. 2025Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Modelscited by 62
  30. 2025One Token to Fool LLM-as-a-Judgecited by 60
  31. 2025Self-Challenging Language Model Agentscited by 59
  32. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 56
  33. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 56
  34. 2025MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environmentcited by 52
  35. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 51
  36. 2024Self-Consistency Preference Optimizationcited by 50
  37. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoningcited by 46
  38. 2025Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGIcited by 36

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 44; 2026: 184.

228 papers.

Sort:

By area:

  1. 2026Reinforcement Learning via Self-Distillationcited by 332
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  4. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 142
  5. 2025Self-Rewarding Vision-Language Model via Reasoning Decompositioncited by 97
  6. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 83
  7. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
  8. 2025Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agentscited by 62
  9. 2025Parallel-R1: Towards Parallel Thinking via Reinforcement Learningcited by 61
  10. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 60
  11. 2025VisPlay: Self-Evolving Vision-Language Models from Imagescited by 47
  12. 2025Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variationcited by 45
38 more
  1. 2026Dr. Zero: Self-Evolving Search Agents without Training Datacited by 44
  2. 2025Language Self-Play For Data-Free Trainingcited by 43
  3. 2025Toward Training Superintelligent Software Agents through Self-Play SWE-RLcited by 40
  4. 2026Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharingcited by 36
  5. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 35
  6. 2026GATES: Self-Distillation under Privileged Context with Consensus Gatingcited by 35
  7. 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loopscited by 35
  8. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 34
  9. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Datacited by 34
  10. 2025CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Modelscited by 29
  11. 2025SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Modelscited by 29
  12. 2025TruthRL: Incentivizing Truthful LLMs via Reinforcement Learningcited by 27
  13. 2026UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervisioncited by 26
  14. 2025Aegis: Automated Error Generation and Attribution for Multi-Agent Systemscited by 22
  15. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
  16. 2026MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Datacited by 21
  17. 2026Structured Scaling of AI Discovery Across Diverse Scientific Domainscited by 21
  18. 2026Recursive Synthesis for Long-Horizon Terminal Taskscited by 19
  19. 2026World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetrycited by 18
  20. 2026Dream-RSI: Recursive Self-Improvement through Evolving Worldscited by 18
  21. 2026Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gaincited by 17
  22. 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learningcited by 17
  23. 2026G-Zero: Self-Play for Open-Ended Generation from Zero Datacited by 17
  24. 2026On-Policy Self-Distillation without Any Supervisioncited by 17
  25. 2026Position: Agentic Evolution is the Path to Evolving LLMscited by 16
  26. 2026Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Taskscited by 15
  27. 2026Verified Critical Step Optimization for LLM Agentscited by 14
  28. 2026EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learningcited by 14
  29. 2025CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoningcited by 13
  30. 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
  31. 2025Propose, Solve, Verify: Self-Play Through Formal Verificationcited by 13
  32. 2026Teaching Models to Teach Themselves: Reasoning at the Edge of Learnabilitycited by 13
  33. 2026Self-Evolving World Models for LLM Agent Planningcited by 13
  34. 2026EnvHarness: Awakening Static Worlds for Agent Learningcited by 13
  35. 2025RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Trainingcited by 12
  36. 2026Agentic Proposing: Enhancing Large Language Model Reasoning via Compositional Skill Synthesiscited by 12
  37. 2026CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolutioncited by 12
  38. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environmentscited by 12

50 most cited of 228 All 228 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution
  2. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
  4. 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning
  5. 2025Effective Learning for Small Reasoning Models: An Empirical Study on 0.5B Reasoning LLMs
  6. 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
  7. 2025Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving
  8. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch