# Survey of Self-Play in Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.02850 · HTML: https://mlsanity.com/paper/2107.02850/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2021.

## Built on by (most cited first, 8 of 8)

- [Search Self-play: Pushing the Frontier of Agent Capability without Supervision](https://mlsanity.com/paper/2510.18821/) (2025)
- [A Survey on Self-play Methods in Reinforcement Learning](https://mlsanity.com/paper/2408.01072/) (2024)
- [From Context to Skills: Can Language Models Learn from Context Skillfully?](https://mlsanity.com/paper/2604.27660/) (2026)
- [On-the-fly Strategy Adaptation for ad-hoc Agent Coordination](https://mlsanity.com/paper/2203.08015/) (2022)
- [SPFT-SQL: Enhancing Large Language Model for Text-to-SQL Parsing by Self-Play Fine-Tuning](https://mlsanity.com/paper/2509.03937/) (2025)
- [Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolbox](https://mlsanity.com/paper/2212.00253/) (2022)
- [A Fairness-Driven Method for Learning Human-Compatible Negotiation Strategies](https://mlsanity.com/paper/2409.18335/) (2024)
- [Competing for pixels: a self-play algorithm for weakly-supervised segmentation](https://mlsanity.com/paper/2405.16628/) (2024)

## Abstract (arXiv)

In reinforcement learning (RL), the term self-play describes a kind of multi-agent learning (MAL) that deploys an algorithm against copies of itself to test compatibility in various stochastic environments. As is typical in MAL, the literature draws heavily from well-established concepts in classical game theory and so this survey quickly reviews some fundamental concepts. In what follows, we present a brief survey of self-play literature, its major themes, criteria, and techniques, and then conclude with an assessment of current shortfalls of the literature as well as suggestions for future directions.