mlsanityData to 7 Oct 2026

A Unified Framework for Factorizing Distributional Value Functions for Multi-Agent Reinforcement Learning

arXiv 2306.02430 ↗First posted 4 Jun 2023cs.MA cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

In fully cooperative multi-agent reinforcement learning (MARL) settings, environments are highly stochastic due to the partial observability of each agent and the continuously changing policies of other agents. To address the above issues, we proposed a unified framework, called DFAC, for integrating distributional RL with value function factorization methods. This framework generalizes expected value function factorization methods to enable the factorization of return distributions. To validate DFAC, we first demonstrate its ability to factorize the value functions of a simple matrix game with stochastic rewards. Then, we perform experiments on all Super Hard maps of the StarCraft Multi-Agent Challenge and six self-designed Ultra Hard maps, showing that DFAC is able to outperform a number of baselines.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 4 Jun 2023.

Builds on

33 of 46 references.

References · 33

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2018On the Spectral Bias of Neural Networkscited by 1,321
  3. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  4. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  6. 2019The StarCraft Multi-Agent Challengecited by 589
  7. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  8. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  9. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  10. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
  11. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  12. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
21 more
  1. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  2. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
  3. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  4. 2019Fully Parameterized Quantile Function for Distributional Reinforcement Learningcited by 87
  5. 2019Learning Nearly Decomposable Value Functions Via Communication Minimizationcited by 85
  6. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
  7. 2019Statistics and Samples in Distributional Reinforcement Learningcited by 62
  8. 2019Distributional Reinforcement Learning for Efficient Explorationcited by 59
  9. 2018Information-Directed Exploration for Deep Reinforcement Learningcited by 52
  10. 2019A Comparative Analysis of Expected and Distributional Reinforcement Learningcited by 47
  11. 2019Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Controlcited by 46
  12. 2018Multi-Agent Common Knowledge Reinforcement Learningcited by 31
  13. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  14. 2020Towards Understanding Cooperative Multi-Agent Q-Learning with Value Factorizationcited by 26
  15. 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
  16. 2019Action Semantics Network: Considering the Effects of Actions in Multiagent Systemscited by 19
  17. 2019Distributional reinforcement learning with linear function approximationcited by 15
  18. 2018QUOTA: The Quantile Option Architecture for Reinforcement Learningcited by 13
  19. 2019Distributional Reward Decomposition for Reinforcement Learningcited by 11
  20. 2020Risk-Sensitive Markov Decision Processes with Combined Metrics of Mean and Variancecited by 8
  21. 2020QR-MIX: Distributional Value Function Factorisation for Cooperative Multi-Agent Reinforcement Learningcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Enhancing Sample Efficiency in Multi-Agent RL with Uncertainty Quantification and Selective Explorationcited by 1