mlsanityData to 7 Oct 2026

Value-Decomposition Networks For Cooperative Multi-Agent Learning

arXiv 1706.05296 ↗First posted 16 Jun 2017cs.AIcited by 490

Not on the map. Why?

Cited by 36 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We study the problem of cooperative multi-agent reinforcement learning with a single joint reward signal. This class of learning problems is difficult because of the often large combined action and observation spaces. In the fully centralized and decentralized approaches, we find the problem of spurious rewards and a phenomenon we call the "lazy agent" problem, which arises due to partial observability. We address these problems by training individual agents with a novel value decomposition network architecture, which learns to decompose the team value function into agent-wise value functions. We perform an experimental evaluation across a range of partially-observable multi-agent domains and show that learning such value-decompositions leads to superior results, in particular when combined with weight sharing, role information and information channels.

Key facts. Cited by 490 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 66; 2022: 85; 2023: 81; 2024: 76; 2025: 106; 2026: 76.

490 papers.

Sort:

By area:

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  2. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  3. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 129
  4. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  5. 2026MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memorycited by 113
  6. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
  7. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  8. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
  9. 2021Off-Belief Learningcited by 72
  10. 2024Simplifying Deep Temporal Difference Learningcited by 71
  11. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  12. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
38 more
  1. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 59
  2. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 57
  3. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  4. 2021Episodic Multi-agent Reinforcement Learning with Curiosity-Driven Explorationcited by 49
  5. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 48
  6. 2024Learning Multi-Agent Communication from Graph Modeling Perspectivecited by 47
  7. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 46
  8. 2023Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularizationcited by 42
  9. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  10. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  11. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  12. 2021Independent Natural Policy Gradient Always Converges in Markov Potential Gamescited by 34
  13. 2021Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Surveycited by 34
  14. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  15. 2022Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregationcited by 34
  16. 2025MAT-Agent: Adaptive Multi-Agent Training Optimizationcited by 34
  17. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  18. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  19. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  20. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  21. 2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 31
  22. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 31
  23. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  24. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  25. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 28
  26. 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 28
  27. 2021Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approachcited by 27
  28. 2021Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learningcited by 25
  29. 2021Multi-Agent MDP Homomorphic Networkscited by 24
  30. 2022Scalable Multi-Agent Model-Based Reinforcement Learningcited by 24
  31. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
  32. 2024Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigationcited by 24
  33. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  34. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  35. 2021A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertisingcited by 22
  36. 2022Solving Continuous Control via Q-learningcited by 21
  37. 2024K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoningcited by 21
  38. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20

50 most cited of 490 All 490 (JSON)