mlsanityData to 14 Oct 2026

DeCOM: Decomposed Policy for Constrained Cooperative Multi-Agent Reinforcement Learning

arXiv 2111.05670 ↗First posted 10 Nov 2021cs.LG cs.MAcited by 1

Not on the map. Why?

Abstract (arXiv)

In recent years, multi-agent reinforcement learning (MARL) has presented impressive performance in various applications. However, physical limitations, budget restrictions, and many other factors usually impose textitconstraints on a multi-agent system (MAS), which cannot be handled by traditional MARL frameworks. Specifically, this paper focuses on constrained MASes where agents work textitcooperatively to maximize the expected team-average return under various constraints on expected team-average costs, and develops a textitconstrained cooperative MARL framework, named DeCOM, for such MASes. In particular, DeCOM decomposes the policy of each agent into two modules, which empowers information sharing among agents to achieve better cooperation. In addition, with such modularization, the training algorithm of DeCOM separates the original constrained optimization into an unconstrained optimization on reward and a constraints satisfaction problem on costs. DeCOM then iteratively solves these problems in a computationally efficient manner, which makes DeCOM highly scalable. We also provide theoretical guarantees on the convergence of DeCOM's policy update algorithm. Finally, we validate the effectiveness of DeCOM with various types of costs in both toy and large-scale (with 500 agents) environments.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 54 of its 73 references that are in the map. First posted on arXiv 10 Nov 2021.

Builds on

54 of 73 references.

References · 54

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  5. 2017Constrained Policy Optimizationcited by 983
  6. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  8. 2019The StarCraft Multi-Agent Challengecited by 597
  9. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  10. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  11. 2018Reward Constrained Policy Optimizationcited by 417
  12. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
38 more
  1. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  2. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  3. 2019Batch Policy Learning under Constraintscited by 244
  4. 2017Learning with Opponent-Learning Awarenesscited by 222
  5. 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
  6. 2020Projection-Based Constrained Policy Optimizationcited by 203
  7. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  8. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 194
  9. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  10. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 175
  11. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 170
  12. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 151
  13. 2020Exploration-Exploitation in Constrained MDPscited by 149
  14. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  15. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 134
  16. 2020Conservative Safety Critics for Explorationcited by 113
  17. 2020Variational Policy Gradient Method for Reinforcement Learning with General Utilitiescited by 111
  18. 2018Inequity aversion improves cooperation in intertemporal social dilemmascited by 106
  19. 2020Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibriumcited by 100
  20. 2019Deep Coordination Graphscited by 98
  21. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 98
  22. 2019Learning to Schedule Communication in Multi-agent Reinforcement Learningcited by 93
  23. 2021Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPscited by 80
  24. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  25. 2020Safe Reinforcement Learning via Curriculum Inductioncited by 74
  26. 2021Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificatescited by 74
  27. 2019Learning Efficient Multi-agent Communication: An Information Bottleneck Approachcited by 61
  28. 2019Reinforcement Learning with Convex Constraintscited by 60
  29. 2019Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learningcited by 57
  30. 2019Convergent Policy Optimization for Safe Reinforcement Learningcited by 57
  31. 2020Learning Individually Inferred Communication for Multi-Agent Cooperationcited by 55
  32. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
  33. 2021Sample-Efficient Learning of Stackelberg Equilibria in General-Sum Gamescited by 40
  34. 2020Constrained Markov Decision Processes via Backward Value Functionscited by 33
  35. 2019CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platformscited by 22
  36. 2019Value Propagation for Decentralized Networked Deep Multi-agent Reinforcement Learningcited by 19
  37. 2020Learning Multi-Agent Coordination for Enhancing Target Coverage in Directional Sensor Networkscited by 19
  38. 2020Balancing Constraints and Rewards with Meta-Gradient D4PGcited by 18

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026α-fair heterogeneous agent reinforcement learningcited by 0