# Mastering Zero-Shot Interactions in Cooperative and Competitive Simultaneous Games

arXiv: https://arxiv.org/abs/2402.03136 · HTML: https://mlsanity.com/paper/2402.03136/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 67 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Population Based Training of Neural Networks](https://mlsanity.com/paper/1711.09846/) (2017)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- ["Other-Play" for Zero-Shot Coordination](https://mlsanity.com/paper/2003.02979/) (2020)
- [Collaborating with Humans without Human Data](https://mlsanity.com/paper/2110.08176/) (2021)
- [Deep Counterfactual Regret Minimization](https://mlsanity.com/paper/1811.00164/) (2018)
- [Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordination](https://mlsanity.com/paper/2112.11701/) (2021)
- [Boltzmann Exploration Done Right](https://mlsanity.com/paper/1705.10257/) (2017)
- [Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biased](https://mlsanity.com/paper/2302.01605/) (2023)
- [PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordination](https://mlsanity.com/paper/2301.06387/) (2023)
- [PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration](https://mlsanity.com/paper/2203.08553/) (2022)
- [Model-Based Opponent Modeling](https://mlsanity.com/paper/2108.01843/) (2021)
- [Are AlphaZero-like Agents Robust to Adversarial Perturbations?](https://mlsanity.com/paper/2211.03769/) (2022)
- [Parameter estimation in softmax decision-making models with linear objective functions](https://mlsanity.com/paper/1502.04635/) (2015)
- [Battlesnake Challenge: A Multi-agent Reinforcement Learning Playground with Human-in-the-loop](https://mlsanity.com/paper/2007.10504/) (2020)
- [Complexity and Algorithms for Exploiting Quantal Opponents in Large Two-Player Games](https://mlsanity.com/paper/2009.14521/) (2020)
- [POLTER: Policy Trajectory Ensemble Regularization for Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2205.11357/) (2022)
- [Convergence of Monte Carlo Tree Search in Simultaneous Move Games](https://mlsanity.com/paper/1310.8613/) (2013)

## Built on by (most cited first, 1 of 1)

- [Multi-Agent Reinforcement Learning for Inverse Design in Photonic Integrated Circuits](https://mlsanity.com/paper/2506.18627/) (2025)

## Abstract (arXiv)

The combination of self-play and planning has achieved great successes in sequential games, for instance in Chess and Go. However, adapting algorithms such as AlphaZero to simultaneous games poses a new challenge. In these games, missing information about concurrent actions of other agents is a limiting factor as they may select different Nash equilibria or do not play optimally at all. Thus, it is vital to model the behavior of the other agents when interacting with them in simultaneous games. To this end, we propose Albatross: AlphaZero for Learning Bounded-rational Agents and Temperature-based Response Optimization using Simulated Self-play. Albatross learns to play the novel equilibrium concept of a Smooth Best Response Logit Equilibrium (SBRLE), which enables cooperation and competition with agents of any playing strength. We perform an extensive evaluation of Albatross on a set of cooperative and competitive simultaneous perfect-information games. In contrast to AlphaZero, Albatross is able to exploit weak agents in the competitive game of Battlesnake. Additionally, it yields an improvement of 37.6% compared to previous state of the art in the cooperative Overcooked benchmark.