mlsanityData to 7 Oct 2026

MAVEN: Multi-Agent Variational Exploration

arXiv 1910.07483 ↗First posted 16 Oct 2019cs.LG stat.MLcited by 181

Not on the map. Why?

Abstract (arXiv)

Centralised training with decentralised execution is an important setting for cooperative deep multi-agent reinforcement learning due to communication constraints during execution and computational tractability in training. In this paper, we analyse value-based methods that are known to have superior performance in complex environments [43]. We specifically focus on QMIX [40], the current state-of-the-art in this domain. We show that the representational constraints on the joint action-values introduced by QMIX and similar methods lead to provably poor exploration and suboptimality. Furthermore, we propose a novel approach called MAVEN that hybridises value and policy-based methods by introducing a latent space for hierarchical control. The value-based agents condition their behaviour on the shared latent variable controlled by a hierarchical policy. This allows MAVEN to achieve committed, temporally extended exploration, which is key to solving complex multi-agent tasks. Our experimental results show that MAVEN achieves significant performance improvements on the challenging SMAC domain [43].

Key facts. Cited by 181 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 43; 2022: 52; 2023: 35; 2024: 20; 2025: 20; 2026: 11.

181 papers.

Sort:

By area:

  1. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  2. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  3. 2021Collaborating with Humans without Human Datacited by 139
  4. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  5. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  6. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  7. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  8. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  9. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 54
  10. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  11. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  12. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
38 more
  1. 2021Learning more skills through optimistic explorationcited by 37
  2. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  3. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  4. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  5. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  6. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 28
  7. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  8. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  9. 2021Unbiased Asymmetric Reinforcement Learning under Partial Observabilitycited by 24
  10. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  11. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
  12. 2021Regularized Softmax Deep Multi-Agent Q-Learningcited by 19
  13. 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agentscited by 19
  14. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  15. 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
  16. 2022Contrastive Identity-Aware Learning for Multi-Agent Value Decompositioncited by 19
  17. 2023De novo Drug Design using Reinforcement Learning with Multiple GPT Agentscited by 19
  18. 2021Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learningcited by 18
  19. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  20. 2022Generalization in Cooperative Multi-Agent Systemscited by 18
  21. 2023A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problemcited by 18
  22. 2022Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversitycited by 17
  23. 2025Leveraging Large Language Models for Effective and Explainable Multi-Agent Credit Assignmentcited by 17
  24. 2021Survey of Recent Multi-Agent Reinforcement Learning Algorithms Utilizing Centralized Trainingcited by 16
  25. 2021Coordinated Proximal Policy Optimizationcited by 16
  26. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  27. 2022LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learningcited by 15
  28. 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14
  29. 2023Adaptive Coordination in Social Embodied Rearrangementcited by 14
  30. 2021Centralizing State-Values in Dueling Networks for Multi-Robot Reinforcement Learning Mapless Navigationcited by 13
  31. 2022Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motionscited by 13
  32. 2023Parameter Sharing with Network Pruning for Scalable Multi-Agent Deep Reinforcement Learningcited by 13
  33. 2023Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agentscited by 13
  34. 2021Cooperative Multi-Agent Transfer Learning with Level-Adaptive Credit Assignmentcited by 12
  35. 2021Adaptable Agent Populations via a Generative Model of Policiescited by 12
  36. 2022Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradientscited by 12
  37. 2023Effective and Stable Role-Based Multi-Agent Collaboration by Structural Information Principlescited by 12
  38. 2023System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learningcited by 12

50 most cited of 181 All 181 (JSON)