Learned Coordination Conventions in Cooperative MARL: Measuring the Translation Gap Between Theory-Informed Roles and Learned Routing
Not on the map. Why?
Abstract (arXiv)
Role-semantic assignments provide priors over how heterogeneous agents may coordinate, but cooperative MARL systems instead settle on conventions through decentralized, non-stationary learning, with no guarantee that the resulting structure matches those priors. We study this translation gap between theory-informed role expectations and learned coordination structure through a diagnostic combining a role-routing matrix, formation sensitivity (Deltamax), and gradient/occlusion attribution across three-role MiniGrid and SMACv2 (Terran) environments. We show that label-conditioned attention produces substantially more concentrated and role-specific routing than flat MLP baselines, remains stable under 3v3--9v9 scaling, transfers zero-shot across team sizes, and is invariant to ally-slot padding. A 5-seed re-evaluation shows partial alignment between learned conventions and designer-specified priors while revealing where small-n noise can manufacture apparent strategic divergence. We present these results as an empirical framework for measuring coordination structure in cooperative MARL rather than as a new equilibrium concept or causal explanation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 23 references that are in the map. First posted on arXiv 28 Jun 2026.
Builds on
21 of 23 references.
References · 21
- 2017Attention Is All You Needcited by 60,361
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
- 2019Attention is not Explanationcited by 929
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
- 2019Attention is not not Explanationcited by 636
- 2019The StarCraft Multi-Agent Challengecited by 597
- 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
- 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
- 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
9 more
- 2020"Other-Play" for Zero-Shot Coordinationcited by 198
- 2019MAVEN: Multi-Agent Variational Explorationcited by 185
- 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 137
- 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
- 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
- 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 116
- 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 87
- 2021Off-Belief Learningcited by 75
- 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
Built on by
Not cited yet.