# Analysing Factorizations of Action-Value Networks for Cooperative Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/1902.07497 · HTML: https://mlsanity.com/paper/1902.07497/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2019.

## Built on by (most cited first, 12 of 12)

- [Context-Aware Sparse Deep Coordination Graphs](https://mlsanity.com/paper/2106.02886/) (2021)
- [MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimization](https://mlsanity.com/paper/2302.10418/) (2023)
- [Self-Organized Polynomial-Time Coordination Graphs](https://mlsanity.com/paper/2112.03547/) (2021)
- [Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.14344/) (2022)
- [HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging](https://mlsanity.com/paper/2102.00824/) (2021)
- [Decomposed Soft Actor-Critic Method for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2104.06655/) (2021)
- [Non-Linear Coordination Graphs](https://mlsanity.com/paper/2211.08404/) (2022)
- [Learning to Share in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2112.08702/) (2021)
- [ReMIX: Regret Minimization for Monotonic Value Function Factorization in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2302.05593/) (2023)
- [Universally Expressive Communication in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2206.06758/) (2022)
- [PAC: Assisted Value Factorisation with Counterfactual Predictions in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2206.11420/) (2022)
- [Action-Graph Policies: Learning Action Co-dependencies in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2602.17009/) (2026)

## Abstract (arXiv)

Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. However, given the lack of theoretical insight, it remains unclear what the employed neural networks are learning, or how we should enhance their learning power to address the problems on which they fail. In this work, we empirically investigate the learning power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results extend those in [4] and quantify how well various approaches can represent the requisite value functions, and help us identify the reasons that can impede good performance, like sparsity of the values or too tight coordination requirements.