mlsanityData to 14 Oct 2026

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning

arXiv 2003.08839 ↗First posted 19 Mar 2020cs.LG cs.MA stat.MLcited by 485

Not on the map. Why?

Cited by 67 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In many real-world settings, a team of agents must coordinate its behaviour while acting in a decentralised fashion. At the same time, it is often possible to train the agents in a centralised fashion where global state information is available and communication constraints are lifted. Learning joint action-values conditioned on extra state information is an attractive way to exploit centralised learning, but the best strategy for then extracting decentralised policies is unclear. Our solution is QMIX, a novel value-based method that can train decentralised policies in a centralised end-to-end fashion. QMIX employs a mixing network that estimates joint action-values as a monotonic combination of per-agent values. We structurally enforce that the joint-action value is monotonic in the per-agent values, through the use of non-negative weights in the mixing network, which guarantees consistency between the centralised and decentralised policies. To evaluate the performance of QMIX, we propose the StarCraft Multi-Agent Challenge (SMAC) as a new benchmark for deep multi-agent reinforcement learning. We evaluate QMIX on a challenging set of SMAC scenarios and show that it significantly outperforms existing multi-agent reinforcement learning methods.

Key facts. Cited by 485 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 25; 2022: 24; 2023: 72; 2024: 90; 2025: 151; 2026: 123.

485 papers.

Sort:

By area:

  1. 2026MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memorycited by 129
  2. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
  3. 2023MindAgent: Emergent Gaming Interactioncited by 88
  4. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 87
  5. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 86
  6. 2024Simplifying Deep Temporal Difference Learningcited by 78
  7. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 75
  8. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  9. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  10. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 61
  11. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 60
  12. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 58
38 more
  1. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 50
  2. 2024Learning Multi-Agent Communication from Graph Modeling Perspectivecited by 48
  3. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaborationcited by 45
  4. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  5. 2025Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learningcited by 39
  6. 2023Controlling Large Language Model-based Agents for Large-Scale Decision-Making: An Actor-Critic Approachcited by 38
  7. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 37
  8. 2025Multi-Agent Coordination across Diverse Applications: A Surveycited by 37
  9. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  10. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
  11. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  12. 2023OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Controlcited by 33
  13. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  14. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 32
  15. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  16. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 30
  17. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  18. 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
  19. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  20. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  21. 2023ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordinationcited by 24
  22. 2024Learning to Cooperate with Humans using Generative Agentscited by 24
  23. 2025Embodied AI: From LLMs to World Modelscited by 24
  24. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  25. 2022Quantum Multi-Agent Meta Reinforcement Learningcited by 23
  26. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 21
  27. 2024Doubly Mild Generalization for Offline Reinforcement Learningcited by 21
  28. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  29. 2021Quasi-Equivalence Discovery for Zero-Shot Emergent Communicationcited by 19
  30. 2021Regularized Softmax Deep Multi-Agent Q-Learningcited by 19
  31. 2022Generalization in Cooperative Multi-Agent Systemscited by 19
  32. 2025Leveraging Large Language Models for Effective and Explainable Multi-Agent Credit Assignmentcited by 18
  33. 2024DNN Partitioning, Task Offloading, and Resource Allocation in Dynamic Vehicular Networks: A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approachcited by 17
  34. 2024Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defensecited by 17
  35. 2024Offline Reinforcement Learning with OOD State Correction and OOD Action Suppressioncited by 17
  36. 2022Distributed Reinforcement Learning for Robot Teams: A Reviewcited by 16
  37. 2022RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on statescited by 16
  38. 2021Divergence-Regularized Multi-Agent Actor-Criticcited by 15

50 most cited of 485 All 485 (JSON)