mlsanityData to 7 Oct 2026

Absolute Zero: Reinforced Self-play Reasoning with Zero Data

arXiv 2505.03335 ↗First posted 6 May 2025cs.LG cs.AI cs.CLcited by 323

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has shown promise in enhancing the reasoning capabilities of large language models by learning directly from outcome-based rewards. Recent RLVR works that operate under the zero setting avoid supervision in labeling the reasoning process, but still depend on manually curated collections of questions and answers for training. The scarcity of high-quality, human-produced examples raises concerns about the long-term scalability of relying on human supervision, a challenge already evident in the domain of language model pretraining. Furthermore, in a hypothetical future where AI surpasses human intelligence, tasks provided by humans may offer limited learning potential for a superintelligent system. To address these concerns, we propose a new RLVR paradigm called Absolute Zero, in which a single model learns to propose tasks that maximize its own learning progress and improves reasoning by solving them, without relying on any external data. Under this paradigm, we introduce the Absolute Zero Reasoner (AZR), a system that self-evolves its training curriculum and reasoning ability by using a code executor to both validate proposed code reasoning tasks and verify answers, serving as an unified source of verifiable reward to guide open-ended yet grounded learning. Despite being trained entirely without external data, AZR achieves overall SOTA performance on coding and mathematical reasoning tasks, outperforming existing zero-setting models that rely on tens of thousands of in-domain human-curated examples. Furthermore, we demonstrate that AZR can be effectively applied across different model scales and is compatible with various model classes.

Key facts. Cited by 323 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 61 of its 90 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

61 of 90 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  2. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  3. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization8 Apr
  4. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks7 Apr
  5. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  7. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  8. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  9. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  10. 2025Process Reinforcement through Implicit Rewards3 Feb
  11. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov

Other references · 50

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  10. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  11. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  12. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
38 more
  1. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  2. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  3. 2017A Survey on Multi-Task Learningcited by 838
  4. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  5. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  6. 2024Self-Rewarding Language Modelscited by 654
  7. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  8. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  9. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  10. 2025DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decompositioncited by 270
  11. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  12. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  13. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  14. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 182
  15. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  16. 2023TACO: Topics in Algorithmic COde generation datasetcited by 150
  17. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  18. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  19. 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 116
  20. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 89
  21. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 80
  22. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  23. 2024Prover-Verifier Games improve legibility of LLM outputscited by 79
  24. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  25. 2024StateFlow: Enhancing LLM Task-Solving through State-Driven Workflowscited by 65
  26. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 54
  27. 2024To Code, or Not To Code? Exploring Impact of Code in Pre-trainingcited by 47
  28. 2024Learning Formal Mathematics From Intrinsic Motivationcited by 45
  29. 2019Regularizing Deep Multi-Task Networks using Orthogonal Gradientscited by 36
  30. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  31. 2023Train Once, Get a Family: State-Adaptive Balances for Offline-to-Online Reinforcement Learningcited by 26
  32. 2024Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editingcited by 23
  33. 2024Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Playcited by 21
  34. 2023Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RLcited by 20
  35. 2019Embedded Agencycited by 16
  36. 2024Boundless Socratic Learning with Language Gamescited by 12
  37. 2024DiveR-CT: Diversity-enhanced Red Teaming Large Language Model Assistants with Relaxing Constraintscited by 10
  38. 2022A Mixture of Surprises for Unsupervised Reinforcement Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 104; 2026: 219.

323 papers.

Sort:

By area:

  1. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  2. 2026Reinforcement Learning via Self-Distillationcited by 285
  3. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  4. 2025Learning to Reason without External Rewardscited by 199
  5. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  6. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  7. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  8. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
  9. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  10. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 129
  11. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
  12. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 89
38 more
  1. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 81
  2. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 78
  3. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 68
  4. 2025Can Large Reasoning Models Self-Train?cited by 66
  5. 2025Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Modelscited by 61
  6. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 58
  7. 2025Self-Challenging Language Model Agentscited by 56
  8. 2025Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agentscited by 52
  9. 2025Self-Questioning Language Modelscited by 47
  10. 2025SciMaster: Towards General-Purpose Scientific AI Agents, Part I. X-Master as Foundation: Can We Lead on Humanity's Last Exam?cited by 45
  11. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  12. 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 43
  13. 2025Language Self-Play For Data-Free Trainingcited by 42
  14. 2025VisPlay: Self-Evolving Vision-Language Models from Imagescited by 42
  15. 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Spacecited by 40
  16. 2026Dr. Zero: Self-Evolving Search Agents without Training Datacited by 39
  17. 2025Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Playcited by 37
  18. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 37
  19. 2025Toward Training Superintelligent Software Agents through Self-Play SWE-RLcited by 37
  20. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  21. 2025The Hallucination Tax of Reinforcement Finetuningcited by 35
  22. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 34
  23. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Modelscited by 33
  24. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 33
  25. 2026Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharingcited by 32
  26. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 31
  27. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Datacited by 31
  28. 2025CoT-Self-Instruct: Building high-quality synthetic prompts for reasoning and non-reasoning taskscited by 30
  29. 2025JarvisEvo: Towards a Self-Evolving Photo Editing Agent with Synergistic Editor-Evaluator Optimizationcited by 30
  30. 2026GATES: Self-Distillation under Privileged Context with Consensus Gatingcited by 30
  31. 2025SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Modelscited by 27
  32. 2026Look Inward to Explore Outward: Learning Temperature Policy from LLM Internal States via Hierarchical RLcited by 27
  33. 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loopscited by 25
  34. 2025RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agentscited by 24
  35. 2026UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervisioncited by 23
  36. 2026Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubricscited by 23
  37. 2025Get Experience from Practice: LLM Agents with Record & Replaycited by 22
  38. 2025Measuring General Intelligence with Generated Gamescited by 21

50 most cited of 323 All 323 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Zero Reinforcement Learning Towards General Domains
  2. 2025Reinforcement Learning is all You Need
  3. 2026The Chase Is the Curriculum, the Capture Anchors the Credit: Pursuit-Evasion Self-Play for Zero-Data LLM Reasoning
  4. 2026DARL: Encouraging Diverse Answers for General Reasoning without Verifiers
  5. 2026Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning
  6. 2026When Self-Belief Misleads: Active Label Acquisition for Reinforcement Learning with Verifiable Rewards
  7. 2026SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions
  8. 2025PretrainZero: Reinforcement Active Pretraining