mlsanityData to 14 Oct 2026

FACMAC: Factored Multi-Agent Centralised Policy Gradients

arXiv 2003.06709 ↗First posted 14 Mar 2020cs.LG cs.AI stat.MLcited by 178

Not on the map. Why?

Cited by 25 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose FACtored Multi-Agent Centralised policy gradients (FACMAC), a new method for cooperative multi-agent reinforcement learning in both discrete and continuous action spaces. Like MADDPG, a popular multi-agent actor-critic method, our approach uses deep deterministic policy gradients to learn policies. However, FACMAC learns a centralised but factored critic, which combines per-agent utilities into the joint action-value function via a non-linear monotonic function, as in QMIX, a popular multi-agent Q-learning algorithm. However, unlike QMIX, there are no inherent constraints on factoring the critic. We thus also employ a nonmonotonic factorisation and empirically demonstrate that its increased representational capacity allows it to solve some tasks that cannot be solved with monolithic, or monotonically factored critics. In addition, FACMAC uses a centralised policy gradient estimator that optimises over the entire joint action space, rather than optimising over each agent's action space separately as in MADDPG. This allows for more coordinated policy changes and fully reaps the benefits of a centralised critic. We evaluate FACMAC on variants of the multi-agent particle environments, a novel multi-agent MuJoCo benchmark, and a challenging set of StarCraft II micromanagement tasks. Empirical results demonstrate FACMAC's superior performance over MADDPG and other baselines on all three domains.

Key facts. Cited by 178 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 28; 2023: 37; 2024: 34; 2025: 43; 2026: 29.

178 papers.

Sort:

By area:

  1. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  2. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 122
  3. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 75
  4. 2023Heterogeneous-Agent Reinforcement Learningcited by 69
  5. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  6. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 68
  7. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  8. 2021Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectificationcited by 61
  9. 2022Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergencecited by 57
  10. 2024An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learningcited by 50
  11. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  12. 2021Settling the Variance of Multi-Agent Policy Gradientscited by 45
38 more
  1. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 37
  2. 2021Multi-Agent Constrained Policy Optimisationcited by 35
  3. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 32
  4. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectivescited by 27
  5. 2024Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithmscited by 26
  6. 2022Solving Continuous Control via Q-learningcited by 24
  7. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
  8. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  9. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 21
  10. 2023LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarioscited by 20
  11. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  12. 2023A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problemcited by 19
  13. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 17
  14. 2021Mava: a research library for distributed multi-agent reinforcement learning in JAXcited by 16
  15. 2022Low-Rank Modular Reinforcement Learning via Muscle Synergycited by 16
  16. 2021Divergence-Regularized Multi-Agent Actor-Criticcited by 15
  17. 2023Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilitiescited by 15
  18. 2023Improving Generalization and Data Efficiency with Diffusion in Offline Multi-agent RLcited by 15
  19. 2022ERL-Re2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representationcited by 14
  20. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learningcited by 13
  21. 2025Robust Gymnasium: A Unified Modular Benchmark for Robust Reinforcement Learningcited by 13
  22. 2022Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARLcited by 12
  23. 2022Learning from Good Trajectories in Offline Multi-Agent Reinforcement Learningcited by 12
  24. 2023Decentralized Monte Carlo Tree Search for Partially Observable Multi-agent Pathfindingcited by 12
  25. 2023Informed POMDP: Leveraging Additional Information in Model-Based RLcited by 11
  26. 2024HyperMARL: Adaptive Hypernetworks for Multi-Agent RLcited by 11
  27. 2023A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Managementcited by 10
  28. 2023AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimationcited by 10
  29. 2024Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Modelscited by 10
  30. 2024On Centralized Critics in Multi-Agent Reinforcement Learningcited by 10
  31. 2024Sable: a Performant, Efficient and Scalable Sequence Model for MARLcited by 10
  32. 2025Multi-agent Coordination via Flow Matchingcited by 10
  33. 2023IMP-MARL: a Suite of Environments for Large-scale Infrastructure Management Planning via MARLcited by 9
  34. 2023FoX: Formation-aware exploration in multi-agent reinforcement learningcited by 9
  35. 2024Multi-Sender Persuasion: A Computational Perspectivecited by 9
  36. 2024Efficient Multi-agent Reinforcement Learning by Planningcited by 9
  37. 2024Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and Evaluationcited by 9
  38. 2024Stop Wandering, Find the Keys: LLMs Discriminate Key States for Efficient Multi-Agent Explorationcited by 9

50 most cited of 178 All 178 (JSON)