# OvercookedV2: Rethinking Overcooked for Zero-Shot Coordination

arXiv: https://arxiv.org/abs/2503.17821 · HTML: https://mlsanity.com/paper/2503.17821/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 22 Mar 2025.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [AI Agents That Matter](https://mlsanity.com/paper/2407.01502/) (2024)
- ["Other-Play" for Zero-Shot Coordination](https://mlsanity.com/paper/2003.02979/) (2020)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [Collaborating with Humans without Human Data](https://mlsanity.com/paper/2110.08176/) (2021)
- [SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2212.07489/) (2022)
- [Few-Shot Preference Learning for Human-in-the-Loop RL](https://mlsanity.com/paper/2212.03363/) (2022)
- [AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents](https://mlsanity.com/paper/2401.12963/) (2024)
- [Off-Belief Learning](https://mlsanity.com/paper/2103.04000/) (2021)
- [Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordination](https://mlsanity.com/paper/2112.11701/) (2021)
- [Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1811.01458/) (2018)
- [Towards a Standardised Performance Evaluation Protocol for Cooperative MARL](https://mlsanity.com/paper/2209.10485/) (2022)
- [LLM-Powered Hierarchical Language Agent for Real-time Human-AI Coordination](https://mlsanity.com/paper/2312.15224/) (2023)
- [Why Warmup the Learning Rate? Underlying Mechanisms and Improvements](https://mlsanity.com/paper/2406.09405/) (2024)
- [Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biased](https://mlsanity.com/paper/2302.01605/) (2023)
- [True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning](https://mlsanity.com/paper/2401.14151/) (2024)
- [Evaluating the Robustness of Collaborative Agents](https://mlsanity.com/paper/2101.05507/) (2021)
- [SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention](https://mlsanity.com/paper/2312.01990/) (2023)
- [Optimal Behavior Prior: Data-Efficient Human Models for Improved Human-AI Collaboration](https://mlsanity.com/paper/2211.01602/) (2022)
- [Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2404.17780/) (2024)

## Built on by (most cited first, 16 of 16)

- [The Decrypto Benchmark for Multi-Agent Reasoning and Theory of Mind](https://mlsanity.com/paper/2506.20664/) (2025)
- [Adaptively Coordinating with Novel Partners via Learned Latent Strategies](https://mlsanity.com/paper/2511.12754/) (2025)
- [The Yokai Learning Environment: Tracking Beliefs Over Space and Time](https://mlsanity.com/paper/2508.12480/) (2025)
- [High entropy leads to symmetry-equivariant policies in Dec-POMDPs](https://mlsanity.com/paper/2511.22581/) (2025)
- [Fatigue-Aware Learning to Defer via Constrained Optimisation](https://mlsanity.com/paper/2604.00904/) (2026)
- [Probing Dec-POMDP Reasoning in Cooperative MARL](https://mlsanity.com/paper/2602.20804/) (2026)
- [A Recipe for Stable Offline Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2603.08399/) (2026)
- [Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork](https://mlsanity.com/paper/2605.24423/) (2026)
- [Benchmarking Open-Ended Multi-Agent Coordination in Language Agents](https://mlsanity.com/paper/2606.08340/) (2026)
- [The challenge of hidden gifts in multi-agent reinforcement learning](https://mlsanity.com/paper/2505.20579/) (2025)
- [Strategically Robust Multi-Agent Reinforcement Learning with Linear Function Approximation](https://mlsanity.com/paper/2603.09208/) (2026)
- [Beyond Partner Diversity: An Influence-Based Team Steering Framework for Zero-Shot Human-Machine Teaming](https://mlsanity.com/paper/2605.15400/) (2026)
- [Shaping Persistent Representations from Independent Interactions](https://mlsanity.com/paper/2609.34604/) (2026)
- [OverForge: Reasoning Through Strategies and Tactics Helps Cooperative Lifelong Adaptation](https://mlsanity.com/paper/2609.39727/) (2026)
- [Partially Observable Zero-shot coordination by Predicting Intention of Partner](https://mlsanity.com/paper/2610.08142/) (2026)
- [MARL-BattleGrounds: A Novel JAX-native Benchmark for Heterogeneous and Competitive Multi-agent RL](https://mlsanity.com/paper/2610.14883/) (2026)

## Abstract (arXiv)

AI agents hold the potential to transform everyday life by helping humans achieve their goals. To do this successfully, agents need to be able to coordinate with novel partners without prior interaction, a setting known as zero-shot coordination (ZSC). Overcooked has become one of the most popular benchmarks for evaluating coordination capabilities of AI agents and learning algorithms. In this work, we investigate the origins of ZSC challenges in Overcooked. We introduce a state augmentation mechanism which mixes states that might be encountered when paired with unknown partners into the training distribution, reducing the out-of-distribution challenge associated with ZSC. We show that independently trained agents under this algorithm coordinate successfully in Overcooked. Our results suggest that ZSC failure can largely be attributed to poor state coverage under self-play rather than more sophisticated coordination challenges. The Overcooked environment is therefore not suitable as a ZSC benchmark. To address these shortcomings, we introduce OvercookedV2, a new version of the benchmark, which includes asymmetric information and stochasticity, facilitating the creation of interesting ZSC scenarios. To validate OvercookedV2, we conduct experiments demonstrating that mere exhaustive state coverage is insufficient to coordinate well. Finally, we use OvercookedV2 to build a new range of coordination challenges, including ones that require test time protocol formation, and we demonstrate the need for new coordination algorithms that can adapt online. We hope that OvercookedV2 will help benchmark the next generation of ZSC algorithms and advance collaboration between AI agents and humans.