# Simplex Neural Population Learning: Any-Mixture Bayes-Optimality in Symmetric Zero-sum Games

arXiv: https://arxiv.org/abs/2205.15879 · HTML: https://mlsanity.com/paper/2205.15879/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 15 of 15)

- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning](https://mlsanity.com/paper/1910.08348/) (2019)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [Open-ended Learning in Symmetric Zero-sum Games](https://mlsanity.com/paper/1901.08106/) (2019)
- [Meta reinforcement learning as task inference](https://mlsanity.com/paper/1905.06424/) (2019)
- [Meta-learning of Sequential Strategies](https://mlsanity.com/paper/1905.03030/) (2019)
- [Modeling Others using Oneself in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.09640/) (2018)
- [Meta-trained agents implement Bayes-optimal agents](https://mlsanity.com/paper/2010.11223/) (2020)
- [Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity](https://mlsanity.com/paper/2110.04041/) (2021)
- [NeuPL: Neural Population Learning](https://mlsanity.com/paper/2202.07415/) (2022)
- [Iterative Empirical Game Solving via Single Policy Best Response](https://mlsanity.com/paper/2106.01901/) (2021)
- [Predicting Human Behavior in Unrepeated, Simultaneous-Move Games](https://mlsanity.com/paper/1306.0918/) (2013)
- [Anytime PSRO for Two-Player Zero-Sum Games](https://mlsanity.com/paper/2201.07700/) (2022)
- [Learning to Play against Any Mixture of Opponents](https://mlsanity.com/paper/2009.14180/) (2020)

## Built on by (most cited first, 8 of 8)

- [A Survey on Self-play Methods in Reinforcement Learning](https://mlsanity.com/paper/2408.01072/) (2024)
- [Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications](https://mlsanity.com/paper/2404.13506/) (2024)
- [Diversifying AI: Towards Creative Chess with AlphaZero](https://mlsanity.com/paper/2308.09175/) (2023)
- [Turbocharging Solution Concepts: Solving NEs, CEs and CCEs with Neural Equilibrium Solvers](https://mlsanity.com/paper/2210.09257/) (2022)
- [Policy Space Response Oracles: A Survey](https://mlsanity.com/paper/2403.02227/) (2024)
- [Are Equivariant Equilibrium Approximators Beneficial?](https://mlsanity.com/paper/2301.11481/) (2023)
- [Breaking the mold: The challenge of large scale MARL specialization](https://mlsanity.com/paper/2410.02128/) (2024)
- [Global Policy-Space Response Oracles for Two-Player Zero-Sum Games](https://mlsanity.com/paper/2605.28273/) (2026)

## Abstract (arXiv)

Learning to play optimally against any mixture over a diverse set of strategies is of important practical interests in competitive games. In this paper, we propose simplex-NeuPL that satisfies two desiderata simultaneously: i) learning a population of strategically diverse basis policies, represented by a single conditional network; ii) using the same network, learn best-responses to any mixture over the simplex of basis policies. We show that the resulting conditional policies incorporate prior information about their opponents effectively, enabling near optimal returns against arbitrary mixture policies in a game with tractable best-responses. We verify that such policies behave Bayes-optimally under uncertainty and offer insights in using this flexibility at test time. Finally, we offer evidence that learning best-responses to any mixture policies is an effective auxiliary task for strategic exploration, which, by itself, can lead to more performant populations.