mlsanityData to 7 Oct 2026

Self-play RL for reasoning

72 papers0 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: self-play, reasoning, reinforcement learning, reinforcement, training

Key facts. Self-play RL for reasoning: 72 papers on arXiv since 2023 (data through 7 Oct 2026). 21 of its 72 mapped papers (29%) cite “Absolute Zero: Reinforced Self-play Reasoning with Zero Data” (2025, arXiv 2505.03335). 12 of its 72 mapped papers (17%) cite “Self-playing Adversarial Language Game Enhances LLM Reasoning” (2024, arXiv 2404.10642). 5 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

5 new papers, 17 Sep – 7 Oct; 0 this week.

  1. Earlier · 5
  2. ROSS: Relearning from Self-Generated Rollouts through Selective Supervision also fits: Self-rewarding alignment and self-training
  3. Teach to Learn: Hint Annealing for Self-improving LLM Reasoning
  4. Learning to Optimize through Solver-Grounded Self-Play
  5. Self-Play Search Distillation for Large Language Model Reasoning
  6. Self-Play Pretraining with Zero Data also fits: Code and instruction data synthesis

Foundations

Most cited by papers here

  1. Absolute Zero: Reinforced Self-play Reasoning with Zero Data2025cited by 21 here
  2. Self-playing Adversarial Language Game Enhances LLM Reasoning2024cited by 12 here
  3. SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning2025cited by 10 here
  4. STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Proving2025cited by 7 here
  5. Language Self-Play For Data-Free Training2025cited by 6 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 13 here
  2. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 4 here
  3. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 7 here
  4. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 16 here
  5. 2024Self-Rewarding Language Modelscited by 11 here
  6. 2024Self-Play Preference Optimization for Language Model Alignmentcited by 7 here
  7. 2024Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Playcited by 4 here
  8. 2025SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoningcited by 5 here
16 more it builds on, not in this chain
  1. 2022Large Language Models Can Self-Improvecited by 3 here
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 4 here
  3. 2023Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedbackcited by 3 here
  4. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 6 here
  5. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 12 here
  6. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 3 here
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 3 here
  8. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 9 here
  9. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 3 here
  10. 2025STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Provingcited by 7 here
  11. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 21 here
  12. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 4 here
  13. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 10 here
  14. 2025Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correctioncited by 3 here
  15. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 16 here
  16. 2025Language Self-Play For Data-Free Trainingcited by 6 here

The 24 most cited of 97.

Browse

30 of 72, ranked 6–35

  1. 6SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning20255
  2. 7SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data20254
  3. 8Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction20253
  4. 9ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search20243
  5. 10Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback20233
  6. 11GASP: Guided Asymmetric Self-Play For Coding LLMs20262
  7. 12Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning20252
  8. 13Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR20252
  9. 14SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models20252
  10. 15Towards Understanding Self-play for LLM Reasoning20252
All 30
  1. 16OpenSIR: Open-Ended Self-Improving Reasoner20252
  2. 17DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data20242
  3. 18Scaling Self-Play with Self-Guidance20261
  4. 19TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning20251
  5. 20Can Large Language Models Play Games? A Case Study of A Self-Play Approach20241
  6. 21TeaMs-RL: Teaching LLMs to Generate Better Instruction Datasets via Reinforcement Learning20241
  7. 22HunyuanProver: A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Proving20241
  8. 23TheoremForge: Scaling up Formal Data Synthesis with Low-Budget Agentic Workflow20260
  9. 24Your Self-Play Algorithm is Secretly an Adversarial Imitator: Understanding LLM Self-Play through the Lens of Imitation Learning20260
  10. 25WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement20260
  11. 26Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning20260
  12. 27Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play20260
  13. 28Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Text20260
  14. 29MathDuels: A Self-Play Benchmark That Grows20260
  15. 30SPARK: Self-Play with Asymmetric Reward from Knowledge Graphs20260
  16. 31Seirenes: Adversarial Self-Play with Evolving Distractions for LLM Reasoning20260
  17. 32Learning to Build the Environment: Self-Evolving Reasoning RL via Verifiable Environment Synthesis20260
  18. 33PopuLoRA: Co-Evolving LLM Populations for Reasoning Self-Play20260
  19. 34One-Way Policy Optimization for Self-Evolving LLMs20260
  20. 35Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL20260