# Self-play RL for reasoning

Area: [Synthetic data and self-improvement for LLMs](https://mlsanity.com/synth/index.md). HTML: https://mlsanity.com/synth/self-play-rl-for-reasoning/

## Key facts

- Self-play RL for reasoning: 72 papers on arXiv since 2023 (data through 7 Oct 2026).
- 21 of its 72 mapped papers (29%) cite “Absolute Zero: Reinforced Self-play Reasoning with Zero Data” (2025, arXiv 2505.03335).
- 12 of its 72 mapped papers (17%) cite “Self-playing Adversarial Language Game Enhances LLM Reasoning” (2024, arXiv 2404.10642).
- 5 new papers 17 Sep–7 Oct.

## Most cited here

- [Absolute Zero: Reinforced Self-play Reasoning with Zero Data](https://mlsanity.com/paper/2505.03335/) (2025), cited by 21 papers here
- [Self-playing Adversarial Language Game Enhances LLM Reasoning](https://mlsanity.com/paper/2404.10642/) (2024), cited by 12 papers here
- [SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2506.24119/) (2025), cited by 10 papers here
- [STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Proving](https://mlsanity.com/paper/2502.00212/) (2025), cited by 7 papers here
- [Language Self-Play For Data-Free Training](https://mlsanity.com/paper/2509.07414/) (2025), cited by 6 papers here
- [SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning](https://mlsanity.com/paper/2504.19162/) (2025), cited by 5 papers here
- [SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data](https://mlsanity.com/paper/2505.20347/) (2025), cited by 4 papers here
- [Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback](https://mlsanity.com/paper/2305.10142/) (2023), cited by 3 papers here
- [ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search](https://mlsanity.com/paper/2406.03816/) (2024), cited by 3 papers here
- [Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction](https://mlsanity.com/paper/2508.03613/) (2025), cited by 3 papers here

## New (17 Sep–7 Oct)

- [ROSS: Relearning from Self-Generated Rollouts through Selective Supervision](https://mlsanity.com/paper/2609.35954/) (2026-09-28)
- [Teach to Learn: Hint Annealing for Self-improving LLM Reasoning](https://mlsanity.com/paper/2609.34975/) (2026-09-28)
- [Learning to Optimize through Solver-Grounded Self-Play](https://mlsanity.com/paper/2609.34205/) (2026-09-28)
- [Self-Play Search Distillation for Large Language Model Reasoning](https://mlsanity.com/paper/2609.30936/) (2026-09-25)
- [Self-Play Pretraining with Zero Data](https://mlsanity.com/paper/2609.30063/) (2026-09-24)