mlsanityData to 14 Oct 2026

Value-Decomposition Multi-Agent Actor-Critics

arXiv 2007.12306 ↗First posted 24 Jul 2020cs.AI cs.LG cs.MAcited by 48

Not on the map. Why?

Abstract (arXiv)

The exploitation of extra state information has been an active research area in multi-agent reinforcement learning (MARL). QMIX represents the joint action-value using a non-negative function approximator and achieves the best performance, by far, on multi-agent benchmarks, StarCraft II micromanagement tasks. However, our experiments show that, in some cases, QMIX is incompatible with A2C, a training paradigm that promotes algorithm training efficiency. To obtain a reasonable trade-off between training efficiency and algorithm performance, we extend value-decomposition to actor-critics that are compatible with A2C and propose a novel actor-critic framework, value-decomposition actor-critics (VDACs). We evaluate VDACs on the testbed of StarCraft II micromanagement tasks and demonstrate that the proposed framework improves median performance over other actor-critic methods. Furthermore, we use a set of ablation experiments to identify the key factors that contribute to the performance of VDACs.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 14; 2023: 6; 2024: 8; 2025: 11; 2026: 4.

48 papers.

Sort:

By area:

  1. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  2. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  3. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  4. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  5. 2022Solving Continuous Control via Q-learningcited by 24
  6. 2022What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?cited by 22
  7. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  8. 2023MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimizationcited by 20
  9. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  10. 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
  11. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  12. 2022Asynchronous Actor-Critic for Multi-Agent Reinforcement Learningcited by 18
36 more
  1. 2021Coordinated Proximal Policy Optimizationcited by 16
  2. 2022LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learningcited by 15
  3. 2022Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradientscited by 12
  4. 2022A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learningcited by 10
  5. 2024On Centralized Critics in Multi-Agent Reinforcement Learningcited by 10
  6. 2025Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learningcited by 10
  7. 2024Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Explorationcited by 9
  8. 2022Interaction Pattern Disentangling for Multi-Agent Reinforcement Learningcited by 8
  9. 2023Global Rewards in Multi-Agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systemscited by 8
  10. 2024Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolutioncited by 7
  11. 2025QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning?cited by 6
  12. 2023A Variational Approach to Mutual Information-Based Coordination for Multi-Agent Reinforcement Learningcited by 5
  13. 2023XuanCe: A Comprehensive and Unified Deep Reinforcement Learning Librarycited by 5
  14. 2024AgentMixer: Multi-Agent Correlated Policy Factorizationcited by 5
  15. 2024Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure managementcited by 5
  16. 2025An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Managementcited by 5
  17. 2025Triple-BERT: Do We Really Need MARL for Order Dispatch on Ride-Sharing Platforms?cited by 4
  18. 2021Local Advantage Actor-Critic for Robust Multi-Agent Deep Reinforcement Learningcited by 3
  19. 2022Learning Explicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning via Polarization Policy Gradientcited by 3
  20. 2023ReMIX: Regret Minimization for Monotonic Value Function Factorization in Multiagent Reinforcement Learningcited by 3
  21. 2025Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Explorationcited by 3
  22. 2026Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learningcited by 3
  23. 2022PAC: Assisted Value Factorisation with Counterfactual Predictions in Multi-Agent Reinforcement Learningcited by 2
  24. 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Surveycited by 2
  25. 2025HLSMAC: A New StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Makingcited by 2
  26. 2026A Recipe for Stable Offline Multi-agent Reinforcement Learningcited by 2
  27. 2026Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensuscited by 2
  28. 2022Hierarchical Strategies for Cooperative Multi-Agent Reinforcement Learningcited by 1
  29. 2025Cooperative Game-Theoretic Credit Assignment for Multi-Agent Policy Gradients via the Corecited by 1
  30. 2024Innate-Values-driven Reinforcement Learning based Cooperative Multi-Agent Cognitive Modelingcited by 0
  31. 2024Prioritized League Reinforcement Learning for Large-Scale Heterogeneous Multiagent Systemscited by 0
  32. 2024Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Searchcited by 0
  33. 2025Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspectivecited by 0
  34. 2025Multi-Agent Cross-Entropy Method with Monotonic Nonlinear Critic Decompositioncited by 0
  35. 2025Cooperative Energy Scheduling of Multi-Microgrids Based on Risk-Sensitive Reinforcement Learningcited by 0
  36. 2026Shared Voxel-Map-Based Cooperative Indoor UAV Guidance with a Multi-Agent Soft Actor-Critic Controllercited by 0