# Value-Decomposition Multi-Agent Actor-Critics

arXiv: https://arxiv.org/abs/2007.12306 · HTML: https://mlsanity.com/paper/2007.12306/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2020.

## Built on by (most cited first, 30 of 48)

- [Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2102.03479/) (2021)
- [Towards a Standardised Performance Evaluation Protocol for Cooperative MARL](https://mlsanity.com/paper/2209.10485/) (2022)
- [Deep Multiagent Reinforcement Learning: Challenges and Directions](https://mlsanity.com/paper/2106.15691/) (2021)
- [Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2206.07505/) (2022)
- [Solving Continuous Control via Q-learning](https://mlsanity.com/paper/2210.12566/) (2022)
- [What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?](https://mlsanity.com/paper/2212.02705/) (2022)
- [MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library](https://mlsanity.com/paper/2210.13708/) (2022)
- [MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimization](https://mlsanity.com/paper/2302.10418/) (2023)
- [HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism](https://mlsanity.com/paper/2110.07246/) (2021)
- [More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorization](https://mlsanity.com/paper/2209.12681/) (2022)
- [Maximum Entropy Heterogeneous-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.10715/) (2023)
- [Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.10113/) (2022)
- [Coordinated Proximal Policy Optimization](https://mlsanity.com/paper/2111.04051/) (2021)
- [LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.02561/) (2022)
- [Value Functions Factorization with Latent State Information Sharing in Decentralized Multi-Agent Policy Gradients](https://mlsanity.com/paper/2201.01247/) (2022)
- [A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2201.01221/) (2022)
- [On Centralized Critics in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2408.14597/) (2024)
- [Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2502.03723/) (2025)
- [Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Exploration](https://mlsanity.com/paper/2410.02511/) (2024)
- [Interaction Pattern Disentangling for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2207.03902/) (2022)
- [Global Rewards in Multi-Agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems](https://mlsanity.com/paper/2312.08884/) (2023)
- [Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution](https://mlsanity.com/paper/2404.04253/) (2024)
- [QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning?](https://mlsanity.com/paper/2504.12961/) (2025)
- [A Variational Approach to Mutual Information-Based Coordination for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.00451/) (2023)
- [XuanCe: A Comprehensive and Unified Deep Reinforcement Learning Library](https://mlsanity.com/paper/2312.16248/) (2023)
- [AgentMixer: Multi-Agent Correlated Policy Factorization](https://mlsanity.com/paper/2401.08728/) (2024)
- [Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure management](https://mlsanity.com/paper/2401.12455/) (2024)
- [An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management](https://mlsanity.com/paper/2501.12991/) (2025)
- [Triple-BERT: Do We Really Need MARL for Order Dispatch on Ride-Sharing Platforms?](https://mlsanity.com/paper/2510.03257/) (2025)
- [Local Advantage Actor-Critic for Robust Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2110.08642/) (2021)

## Abstract (arXiv)

The exploitation of extra state information has been an active research area in multi-agent reinforcement learning (MARL). QMIX represents the joint action-value using a non-negative function approximator and achieves the best performance, by far, on multi-agent benchmarks, StarCraft II micromanagement tasks. However, our experiments show that, in some cases, QMIX is incompatible with A2C, a training paradigm that promotes algorithm training efficiency. To obtain a reasonable trade-off between training efficiency and algorithm performance, we extend value-decomposition to actor-critics that are compatible with A2C and propose a novel actor-critic framework, value-decomposition actor-critics (VDACs). We evaluate VDACs on the testbed of StarCraft II micromanagement tasks and demonstrate that the proposed framework improves median performance over other actor-critic methods. Furthermore, we use a set of ablation experiments to identify the key factors that contribute to the performance of VDACs.