Zero-Shot Coordination in Ad Hoc Teams with Generalized Policy Improvement and Difference Rewards
Not on the map. Why?
Abstract (arXiv)
Real-world multi-agent systems may require ad hoc teaming, where an agent must coordinate with other previously unseen teammates to solve a task in a zero-shot manner. Prior work often either selects a pretrained policy based on an inferred model of the new teammates or pretrains a single policy that is robust to potential teammates. Instead, we propose to leverage all pretrained policies in a zero-shot transfer setting. We formalize this problem as an ad hoc multi-agent Markov decision process and present a solution that uses two key ideas, generalized policy improvement and difference rewards, for efficient and effective knowledge transfer between different teams. We empirically demonstrate that our algorithm, Generalized Policy improvement for Ad hoc Teaming (GPAT), successfully enables zero-shot transfer to new teams in three simulated environments: cooperative foraging, predator-prey, and Overcooked. We also demonstrate our algorithm in a real-world multi-robot setting.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 38 references that are in the map. First posted on arXiv 17 Oct 2025.
Builds on
23 of 38 references.
References · 23
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
- 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
- 2016Successor Features for Transfer in Reinforcement Learningcited by 340
- 2020"Other-Play" for Zero-Shot Coordinationcited by 198
- 2021Collaborating with Humans without Human Datacited by 142
- 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 133
- 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
- 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 70
- 2022A Survey of Ad Hoc Teamwork Researchcited by 43
- 2020Towards Open Ad Hoc Teamwork Using Graph-based Policy Learningcited by 37
- 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
11 more
- 2022The Utility of Explainable AI in Ad Hoc Human-Machine Teamingcited by 30
- 2023ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordinationcited by 24
- 2022Any-Play: An Intrinsic Augmentation for Zero-Shot Coordinationcited by 23
- 2024N-Agent Ad Hoc Teamworkcited by 15
- 2022Coordination with Humans via Strategy Matchingcited by 14
- 2023Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabicited by 10
- 2020Difference Rewards Policy Gradientscited by 8
- 2022On-the-fly Strategy Adaptation for ad-hoc Agent Coordinationcited by 7
- 2022Assisting Unknown Teammates in Unknown Tasks: Ad Hoc Teamwork under Partial Observabilitycited by 6
- 2023Controlling Type Confounding in Ad Hoc Teamwork with Instance-wise Teammate Feedback Rectificationcited by 2
- 2021Adaptive Agent Architecture for Real-time Human-Agent Teamingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.