mlsanityData to 7 Oct 2026

Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing

arXiv 2102.07475 ↗First posted 15 Feb 2021cs.MA cs.LGcited by 85

Not on the map. Why?

Abstract (arXiv)

Sharing parameters in multi-agent deep reinforcement learning has played an essential role in allowing algorithms to scale to a large number of agents. Parameter sharing between agents significantly decreases the number of trainable parameters, shortening training times to tractable levels, and has been linked to more efficient learning. However, having all agents share the same parameters can also have a detrimental effect on learning. We demonstrate the impact of parameter sharing methods on training speed and converged returns, establishing that when applied indiscriminately, their effectiveness is highly dependent on the environment. We propose a novel method to automatically identify agents which may benefit from sharing parameters by partitioning them based on their abilities and goals. Our approach combines the increased sample efficiency of parameter sharing with the representational capacity of multiple independent networks to reduce training time and increase final returns.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 14 of its 27 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

14 of 27 references.

References · 14

  1. 2013Auto-Encoding Variational Bayescited by 3,980
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  6. 2019The StarCraft Multi-Agent Challengecited by 589
  7. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  8. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  9. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  10. 2017Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problemscited by 126
  11. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 114
  12. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 90
2 more
  1. 2015A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systemscited by 41
  2. 2020Succinct and Robust Multi-Agent Communication With Temporal Message Controlcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 15; 2023: 20; 2024: 19; 2025: 19; 2026: 10.

85 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  3. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  4. 2024CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamicscited by 43
  5. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
  6. 2022Scalable Multi-Agent Reinforcement Learning for Warehouse Logistics with Robotic and Human Co-Workerscited by 28
  7. 2022Solving Continuous Control via Q-learningcited by 21
  8. 2022Contrastive Identity-Aware Learning for Multi-Agent Value Decompositioncited by 19
  9. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  10. 2022LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learningcited by 15
  11. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  12. 2024N-Agent Ad Hoc Teamworkcited by 15
38 more
  1. 2022Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learningcited by 14
  2. 2023Parameter Sharing with Network Pruning for Scalable Multi-Agent Deep Reinforcement Learningcited by 13
  3. 2023System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learningcited by 12
  4. 2022Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RLcited by 11
  5. 2022Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learningcited by 10
  6. 2023Multi-agent Continual Coordination via Progressive Task Contextualizationcited by 10
  7. 2024Controlling Behavioral Diversity in Multi-Agent Reinforcement Learningcited by 10
  8. 2024Sable: a Performant, Efficient and Scalable Sequence Model for MARLcited by 10
  9. 2024HyperMARL: Adaptive Hypernetworks for Multi-Agent RLcited by 10
  10. 2023ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learningcited by 9
  11. 2025HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistantscited by 9
  12. 2023Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generationcited by 6
  13. 2023Adaptive parameter sharing for multi-agent reinforcement learningcited by 6
  14. 2024Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaborationcited by 6
  15. 2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 6
  16. 2023Ensemble Value Functions for Efficient Exploration in Multi-Agent Reinforcement Learningcited by 5
  17. 2023An Empirical Study on Google Research Football Multi-agent Scenarioscited by 5
  18. 2024Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learningcited by 5
  19. 2024Crafting Desirable Climate Trajectories with RL Explored Socio-Environmental Simulationscited by 4
  20. 2025Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learningcited by 4
  21. 2025Scalable UAV Multi-Hop Networking via Multi-Agent Reinforcement Learning with Large Language Modelscited by 4
  22. 2022The Multi-Agent Pickup and Delivery Problem: MAPF, MARL and Its Warehouse Applicationscited by 3
  23. 2022Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysiscited by 3
  24. 2022Deep Reinforcement Learning for Multi-Agent Interactioncited by 3
  25. 2023Robust Multi-agent Communication via Multi-view Message Certificationcited by 3
  26. 2024Fully Independent Communication in Multi-Agent Reinforcement Learningcited by 3
  27. 2024Knowing What Not to Do: Leverage Language Model Insights for Action Space Pruning in Multi-agent Reinforcement Learningcited by 3
  28. 2024Dashing for the Golden Snitch: Multi-Drone Time-Optimal Motion Planning with Multi-Agent Reinforcement Learningcited by 3
  29. 2025HAD-Gen: Human-like and Diverse Driving Behavior Modeling for Controllable Scenario Generationcited by 3
  30. 2025Multi-Agent Guided Policy Optimizationcited by 3
  31. 2023Efficient Communication via Self-supervised Information Aggregation for Online and Offline Multi-agent Reinforcement Learningcited by 2
  32. 2023Revisiting the Gumbel-Softmax in MADDPGcited by 2
  33. 2023Discovering Causality for Efficient Cooperation in Multi-Agent Environmentscited by 2
  34. 2023Decentralized Motor Skill Learning for Complex Robotic Systemscited by 2
  35. 2023MAAIP: Multi-Agent Adversarial Interaction Priors for imitation from fighting demonstrations for physics-based characterscited by 2
  36. 2024Multi-Agent Reinforcement Learning for Joint Police Patrol and Dispatchcited by 2
  37. 2024On the Convergence Rates of Federated Q-Learning across Heterogeneous Environmentscited by 2
  38. 2025Capability-Aware Shared Hypernetworks for Flexible Heterogeneous Multi-Robot Coordinationcited by 2

50 most cited of 85 All 85 (JSON)