mlsanityData to 14 Oct 2026

PC3D: Zero-Shot Cooperation Across Variable Rosters via Personalized Context Distillation

arXiv 2605.10377 ↗First posted 11 May 2026cs.LG cs.MAcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Cooperative multi-agent reinforcement learning often assumes a fixed execution team, yet many decentralized systems must operate with varying numbers of active agents during deployment. We study this setting under episodic roster variation: each episode is executed by a set of homogeneous agents, with the team size varying across episodes. Agents act only from local histories, without execution-time communication, privileged coordinators, or online retraining. Therefore, effective cooperation requires each agent to recover relevant context about the active team and adapt its behavior accordingly. To this end, we propose PC3D (Personalized Central Coordination Context Distillation), a method for training decentralized policies to recover and use personalized coordination context from local interaction histories. During training, a set-structured centralized teacher compresses the active team into coordination tokens and personalizes them into agent-specific contexts, which are distilled into decentralized policies. At execution, each agent predicts its own context from local history and adaptively uses it to condition decision-making. Across three cooperative MARL benchmarks, PC3D achieves higher returns than the evaluated baselines with both seen and unseen roster sizes, and ablations attribute these gains to both context distillation and adaptive context use.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 37 of its 48 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

37 of 48 references.

References · 37

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  5. 2017Deep Setscited by 1,421
  6. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  7. 2016HyperNetworkscited by 1,027
  8. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  9. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  10. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  11. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  12. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
25 more
  1. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  2. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
  3. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  4. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
  5. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 220
  6. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 212
  7. 2018Understanding the impact of entropy on policy optimizationcited by 176
  8. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  9. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 116
  10. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  11. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 50
  12. 2020Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learningcited by 48
  13. 2019Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Criticscited by 47
  14. 2019PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learningcited by 47
  15. 2019Learning Transferable Cooperative Behavior in Multi-Agent Teamscited by 44
  16. 2023A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environmentcited by 39
  17. 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
  18. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 33
  19. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 30
  20. 2019Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advancescited by 23
  21. 2022A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learningcited by 12
  22. 2022PTDE: Personalized Training with Distilled Execution for Multi-Agent Reinforcement Learningcited by 11
  23. 2022CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learningcited by 7
  24. 2024Open Ad Hoc Teamwork with Cooperative Game Theorycited by 6
  25. 2025URB -- Urban Routing Benchmark for RL-equipped Connected Autonomous Vehiclescited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fully Decentralized Cooperative Multi-Agent Reinforcement Learning is A Context Modeling Problem
  2. 2025Multi-agent In-context Coordination via Decentralized Memory Retrieval
  3. 2025From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demos
  4. 2024Beyond Joint Demonstrations: Personalized Expert Guidance for Efficient Multi-Agent Reinforcement Learning
  5. 2025Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination
  6. 2026Multi-agent cooperation through in-context co-player inference
  7. 2024Multi-agent cooperation through learning-aware policy gradients
  8. 2025Automata-Conditioned Cooperative Multi-Agent Reinforcement Learning