mlsanityData to 14 Oct 2026

Maximum Correntropy Value Decomposition for Multi-agent Deep Reinforcemen Learning

arXiv 2208.03663 ↗First posted 7 Aug 2022cs.MA cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We explore value decomposition solutions for multi-agent deep reinforcement learning in the popular paradigm of centralized training with decentralized execution(CTDE). As the recognized best solution to CTDE, Weighted QMIX is cutting-edge on StarCraft Multi-agent Challenge (SMAC), with a weighting scheme implemented on QMIX to place more emphasis on the optimal joint actions. However, the fixed weight requires manual tuning according to the application scenarios, which painfully prevents Weighted QMIX from being used in broader engineering applications. In this paper, we first demonstrate the flaw of Weighted QMIX using an ordinary One-Step Matrix Game (OMG), that no matter how the weight is chosen, Weighted QMIX struggles to deal with non-monotonic value decomposition problems with a large variance of reward distributions. Then we characterize the problem of value decomposition as an Underfitting One-edged Robust Regression problem and make the first attempt to give a solution to the value decomposition problem from the perspective of information-theoretical learning. We introduce the Maximum Correntropy Criterion (MCC) as a cost function to dynamically adapt the weight to eliminate the effects of minimum in reward distributions. We simplify the implementation and propose a new algorithm called MCVD. A preliminary experiment conducted on OMG shows that MCVD could deal with non-monotonic value decomposition problems with a large tolerance of kernel bandwidth selection. Further experiments are carried out on Cooperative-Navigation and multiple SMAC scenarios, where MCVD exhibits unprecedented ease of implementation, broad applicability, and stability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 43 references that are in the map. First posted on arXiv 7 Aug 2022.

Builds on

29 of 43 references.

References · 29

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  7. 2016HyperNetworkscited by 1,027
  8. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  9. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  10. 2019The StarCraft Multi-Agent Challengecited by 597
  11. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  12. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
17 more
  1. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  2. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
  3. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  4. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  5. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  6. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
  7. 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 127
  8. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  9. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
  10. 2021Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learningcited by 68
  11. 2015Increasing the Action Gap: New Operators for Reinforcement Learningcited by 49
  12. 2021Multi-Agent Constrained Policy Optimisationcited by 35
  13. 2019SMIX(λ): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learningcited by 15
  14. 2022UNMAS: Multi-Agent Reinforcement Learning for Unshaped Cooperative Scenarioscited by 9
  15. 2016Robust Learning with Kernel Mean p-Power Error Losscited by 4
  16. 2022Residual Q-Networks for Value Function Factorizing in Multi-Agent Reinforcement Learningcited by 3
  17. 2021SA-MATD3:Self-attention-based multi-agent continuous control method in cooperative environmentscited by 2

Built on by

Not cited yet.