mlsanityData to 14 Oct 2026

Unsupervised Partner Design Enables Robust Ad-hoc Teamwork

arXiv 2508.06336 ↗First posted 8 Aug 2025cs.LG cs.AI cs.HCcited by 3

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

We introduce Unsupervised Partner Design (UPD), a population-free multi-agent reinforcement learning method for robust ad-hoc teamwork. UPD generates training partners on-the-fly and selects them adaptively based on a learnability criterion, removing the need for pre-trained partner populations or manual parameter tuning. We show that this simple mechanism enables effective partner diversity and can be extended to joint partner-environment selection when a procedural level generator is available. Across Level-Based Foraging, Overcooked-AI, and the Overcooked Generalisation Challenge, UPD consistently achieves strong performance compared to both population-based and population-free baselines. In a human-AI user study, agents trained with UPD achieve higher returns and are rated as more adaptive, more human-like, and less frustrating than all evaluated baseline methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 40 of its 58 references that are in the map. First posted on arXiv 8 Aug 2025.

Builds on

40 of 58 references.

References · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  3. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  6. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  7. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  8. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  9. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  10. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 212
  11. 2020"Other-Play" for Zero-Shot Coordinationcited by 198
  12. 2020Prioritized Level Replaycited by 198
28 more
  1. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
  2. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  3. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  4. 2021Collaborating with Humans without Human Datacited by 142
  5. 2021Replay-Guided Adversarial Environment Designcited by 111
  6. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  7. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 70
  8. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 58
  9. 2015A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systemscited by 44
  10. 2022A Survey of Ad Hoc Teamwork Researchcited by 43
  11. 2020Towards Open Ad Hoc Teamwork Using Graph-based Policy Learningcited by 37
  12. 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
  13. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 31
  14. 2023Cooperative Open-ended Learning Framework for Zero-shot Coordinationcited by 30
  15. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
  16. 2024Learning to Cooperate with Humans using Generative Agentscited by 24
  17. 2025Learning to Reason at the Frontier of Learnabilitycited by 23
  18. 2021Quantifying the effects of environment and population diversity in multi-agent reinforcement learningcited by 22
  19. 2015Belief and Truth in Hypothesised Behaviourscited by 21
  20. 2023Proximal Curriculum for Reinforcement Learning Agentscited by 20
  21. 2024Refining Minimax Regret for Unsupervised Environment Designcited by 18
  22. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  23. 2025Cross-environment Cooperation Enables Zero-shot Multi-agent Coordinationcited by 14
  24. 2025An Optimisation Framework for Unsupervised Environment Designcited by 10
  25. 2023Generalization through Diversity: Improving Unsupervised Environment Designcited by 9
  26. 2025ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamworkcited by 6
  27. 2025Improving Human-AI Coordination through Online Adversarial Training and Generative Modelscited by 3
  28. 2025Automatic Curriculum Design for Zero-Shot Human-AI Coordinationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025The Yokai Learning Environment: Tracking Beliefs Over Space and Timecited by 3
  2. 2025Robust and Diverse Multi-Agent Learning via Rational Policy Gradientcited by 2
  3. 2026PACE: Parameter Change for Unsupervised Environment Designcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design
  2. 2026Training Generalizable Collaborative Agents via Strategic Risk Aversion
  3. 2026Partner-Aware Hierarchical Skill Discovery for Robust Human-AI Collaboration
  4. 2025Seldonian Reinforcement Learning for Ad Hoc Teamwork
  5. 2025Implicitly Aligning Humans and Autonomous Agents through Shared Task Abstractions
  6. 2026Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork
  7. 2025Generalizable Agent Modeling for Agent Collaboration-Competition Adaptation with Multi-Retrieval and Dynamic Generation
  8. 2026Adaptive Human-AI Coordination via Hierarchical Action Disentanglement