mlsanityData to 14 Oct 2026

Zero Shot Coordination for Sparse Reward Tasks with Diverse Reward Shapings

arXiv 2604.25076 ↗First posted 28 Apr 2026cs.LGcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Many Multi-Agent Reinforcement Learning (MARL) agents fail to adapt properly to cooperating with agents trained with the same objectives but different seeds, algorithms, or other training differences. This is the problem of Zero-Shot Coordination (ZSC), which focuses on training agents to cooperate well with unknown agents. ZSC has been studied for a variety of tabular cases and simple games such as Hanabi, achieving excellent results. However, existing solutions to ZSC only consider identical rewards for your trained agents and all future partners. This is not realistic for the trained agents, as they do not consider the problem of cooperating with agents that have identical sparse objectives but shape the rewards for those objectives in different manner. To address this issue, we show how to train an ensemble of methods using randomized reward shapings chosen using 4 selection algorithms. Experiments done on the Overcooked environment demonstrate consistent improvements of 62.2%-119.2% in sparse reward over baseline ZSC algorithms when playing with agents that have identical sparse rewards but different reward shapings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 20 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

9 of 20 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications22 Jul

Other references · 8

  1. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  2. 2018Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 68
  3. 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
  4. 2020Learning to Play Sequential Games versus Unknown Opponentscited by 18
  5. 2022Heterogeneous Multi-agent Zero-Shot Coordination by Coevolutioncited by 12
  6. 2021Theory of Mind for Deep Reinforcement Learning in Hanabicited by 11
  7. 2022Learning to Play Trajectory Games Against Opponents with Unknown Objectivescited by 11
  8. 2022On-the-fly Strategy Adaptation for ad-hoc Agent Coordinationcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details
  2. 2025Efficient Reinforcement Learning for Zero-Shot Coordination in Evolving Games
  3. 2025Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination
  4. 2024Noisy Zero-Shot Coordination: Breaking The Common Knowledge Assumption In Zero-Shot Coordination Games
  5. 2025A Unified Framework for Zero-Shot Reinforcement Learning
  6. 2024KnowPC: Knowledge-Driven Programmatic Reinforcement Learning for Zero-shot Coordination
  7. 2026Theory of Mind Guided Strategy Adaptation for Zero-Shot Coordination
  8. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodings