mlsanityData to 14 Oct 2026

Celebrating Diversity in Shared Multi-Agent Reinforcement Learning

arXiv 2106.02195 ↗First posted 4 Jun 2021cs.LGcited by 98

Not on the map. Why?

Abstract (arXiv)

Recently, deep multi-agent reinforcement learning (MARL) has shown the promise to solve complex cooperative tasks. Its success is partly because of parameter sharing among agents. However, such sharing may lead agents to behave similarly and limit their coordination capacity. In this paper, we aim to introduce diversity in both optimization and representation of shared multi-agent reinforcement learning. Specifically, we propose an information-theoretical regularization to maximize the mutual information between agents' identities and their trajectories, encouraging extensive exploration and diverse individualized behaviors. In representation, we incorporate agent-specific modules in the shared neural network architecture, which are regularized by L1-norm to promote learning sharing among agents while keeping necessary diversity. Empirical results show that our method achieves state-of-the-art performance on Google Research Football and super hard StarCraft II micromanagement tasks.

Key facts. Cited by 98 papers in the map (data through 14 Oct 2026). Builds on 35 of its 50 references that are in the map. First posted on arXiv 4 Jun 2021.

Builds on

35 of 50 references.

References · 35

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  3. 2018Exploration by Random Network Distillationcited by 993
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  6. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  7. 2019The StarCraft Multi-Agent Challengecited by 597
  8. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  9. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  10. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  11. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  12. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
23 more
  1. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  2. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  3. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  4. 2019MAVEN: Multi-Agent Variational Explorationcited by 185
  5. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 137
  6. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  7. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 122
  8. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 121
  9. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 116
  10. 2019Deep Coordination Graphscited by 98
  11. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 98
  12. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 87
  13. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 86
  14. 2019Influence-Based Multi-Agent Explorationcited by 75
  15. 2019Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learningcited by 68
  16. 2017Guided Deep Reinforcement Learning for Swarm Systemscited by 44
  17. 2020Emergent Social Learning via Multi-agent Reinforcement Learningcited by 44
  18. 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
  19. 2019Neighborhood Cognition Consistent Multi-Agent Reinforcement Learningcited by 17
  20. 2020A Q-values Sharing Framework for Multiagent Reinforcement Learning under Budget Constraintcited by 7
  21. 2021Modeling the Interaction between Agents in Cooperative Multi-Agent Reinforcement Learningcited by 7
  22. 2020Parallel Knowledge Transfer in Multi-Agent Reinforcement Learningcited by 6
  23. 2020Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Searchcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 16; 2023: 24; 2024: 22; 2025: 19; 2026: 14.

98 papers.

Sort:

By area:

  1. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 79
  2. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  3. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 36
  4. 2024PolyNet: Learning Diverse Solution Strategies for Neural Combinatorial Optimizationcited by 35
  5. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  6. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
  7. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
  8. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 20
  9. 2022Contrastive Identity-Aware Learning for Multi-Agent Value Decompositioncited by 19
  10. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  11. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  12. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
38 more
  1. 2022LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learningcited by 15
  2. 2022ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependencycited by 15
  3. 2023System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learningcited by 14
  4. 2024Controlling Behavioral Diversity in Multi-Agent Reinforcement Learningcited by 11
  5. 2024HyperMARL: Adaptive Hypernetworks for Multi-Agent RLcited by 11
  6. 2022Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learningcited by 10
  7. 2022Non-Linear Coordination Graphscited by 10
  8. 2023Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrationscited by 9
  9. 2023FoX: Formation-aware exploration in multi-agent reinforcement learningcited by 9
  10. 2024Multi-Sender Persuasion: A Computational Perspectivecited by 9
  11. 2022Interaction Pattern Disentangling for Multi-Agent Reinforcement Learningcited by 8
  12. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  13. 2022Breaking the Curse of Dimensionality in Multiagent State Space: A Unified Agent Permutation Frameworkcited by 6
  14. 2023Learning Diverse Risk Preferences in Population-based Self-playcited by 6
  15. 2024Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharingcited by 6
  16. 2024Heterogeneous Multi-Agent Reinforcement Learning for Zero-Shot Scalable Collaborationcited by 6
  17. 2024Kaleidoscope: Learnable Masks for Heterogeneous Multi-agent Reinforcement Learningcited by 6
  18. 2022Towards Global Optimality in Cooperative MARL with the Transformation And Distillation Frameworkcited by 5
  19. 2023Ensemble Value Functions for Efficient Exploration in Multi-Agent Reinforcement Learningcited by 5
  20. 2023An Empirical Study on Google Research Football Multi-agent Scenarioscited by 5
  21. 2024Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learningcited by 5
  22. 2024The Bandit Whisperer: Communication Learning for Restless Banditscited by 5
  23. 2025Predicting Multi-Agent Specialization via Task Parallelizabilitycited by 5
  24. 2023Towards Skilled Population Curriculum for Multi-Agent Reinforcement Learningcited by 4
  25. 2023Conditionally Optimistic Exploration for Cooperative Deep Multi-Agent Reinforcement Learningcited by 4
  26. 2023Mix-ME: Quality-Diversity for Multi-Agent Learningcited by 4
  27. 2024Multi-Agent Diagnostics for Robustness via Illuminated Diversitycited by 4
  28. 2024Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learningcited by 4
  29. 2024Novelty-Guided Data Reuse for Efficient and Diversified Multi-Agent Reinforcement Learningcited by 4
  30. 2024The impact of behavioral diversity in multi-agent reinforcement learningcited by 4
  31. 2025Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learningcited by 4
  32. 2025Enhancing Diversity in Parallel Agents: A Maximum State Entropy Exploration Storycited by 4
  33. 2025Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibrationcited by 4
  34. 2023Iteratively Learn Diverse Strategies with State Distance Informationcited by 3
  35. 2025Capability-Aware Shared Hypernetworks for Flexible Heterogeneous Multi-Robot Coordinationcited by 3
  36. 2025Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Explorationcited by 3
  37. 2023Policy Diversity for Cooperative Agentscited by 2
  38. 2024Generalization of Heterogeneous Multi-Robot Policies via Awareness and Communication of Capabilitiescited by 2

50 most cited of 98 All 98 (JSON)