mlsanityData to 7 Oct 2026

QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning

arXiv 1803.11485 ↗First posted 30 Mar 2018cs.LG cs.MA stat.MLcited by 697

Not on the map. Why?

Cited by 34 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or laboratory setting, where global state information is available and communication constraints are lifted. Learning joint action-values conditioned on extra state information is an attractive way to exploit centralised learning, but the best strategy for then extracting decentralised policies is unclear. Our solution is QMIX, a novel value-based method that can train decentralised policies in a centralised end-to-end fashion. QMIX employs a network that estimates joint action-values as a complex non-linear combination of per-agent values that condition only on local observations. We structurally enforce that the joint-action value is monotonic in the per-agent values, which allows tractable maximisation of the joint action-value in off-policy learning, and guarantees consistency between the centralised and decentralised policies. We evaluate QMIX on a challenging set of StarCraft II micromanagement tasks, and show that QMIX significantly outperforms existing value-based multi-agent reinforcement learning methods.

Key facts. Cited by 697 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 140; 2022: 164; 2023: 131; 2024: 82; 2025: 89; 2026: 90.

697 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  3. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  4. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  5. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 125
  6. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  7. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
  8. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  9. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  10. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 97
  11. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  12. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 85
38 more
  1. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
  2. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 78
  3. 2021Off-Belief Learningcited by 72
  4. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  5. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  6. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 59
  7. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  8. 2021Cooperative Exploration for Multi-Agent Deep Reinforcement Learningcited by 57
  9. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  10. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 57
  11. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 54
  12. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  13. 2021The Power of Exploiter: Provable Multi-Agent RL in Large State Spacescited by 48
  14. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  15. 2022Sound Adversarial Audio-Visual Navigationcited by 46
  16. 2021Settling the Variance of Multi-Agent Policy Gradientscited by 45
  17. 2023Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularizationcited by 42
  18. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 41
  19. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  20. 2025Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learningcited by 39
  21. 2021Shapley Counterfactual Credits for Multi-Agent Reinforcement Learningcited by 37
  22. 2022Melting Pot 2.0cited by 37
  23. 2023Learning Zero-Shot Cooperation with Humans, Assuming Humans Are Biasedcited by 37
  24. 2022Multi-agent Dynamic Algorithm Configurationcited by 36
  25. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 36
  26. 2021On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learningcited by 35
  27. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  28. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
  29. 2021Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Surveycited by 34
  30. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  31. 2022Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregationcited by 34
  32. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
  33. 2025MAT-Agent: Adaptive Multi-Agent Training Optimizationcited by 34
  34. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 33
  35. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  36. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  37. 2021Offline Decentralized Multi-Agent Reinforcement Learningcited by 32
  38. 2021Learning Selective Communication for Multi-Agent Path Findingcited by 32

50 most cited of 697 All 697 (JSON)