mlsanityData to 7 Oct 2026

Mean-Field Approximation of Cooperative Constrained Multi-Agent Reinforcement Learning (CMARL)

arXiv 2209.07437 ↗First posted 15 Sep 2022cs.LG cs.MAcited by 5

Not on the map. Why?

Abstract (arXiv)

Mean-Field Control (MFC) has recently been proven to be a scalable tool to approximately solve large-scale multi-agent reinforcement learning (MARL) problems. However, these studies are typically limited to unconstrained cumulative reward maximization framework. In this paper, we show that one can use the MFC approach to approximate the MARL problem even in the presence of constraints. Specifically, we prove that, an N-agent constrained MARL problem, with state, and action spaces of each individual agents being of sizes |X|, and |U| respectively, can be approximated by an associated constrained MFC problem with an error, etriangleq Oleft([√|X|+√|U|]/√Nright). In a special case where the reward, cost, and state transition functions are independent of the action distribution of the population, we prove that the error can be improved to e=O(√|X|/√N). Also, we provide a Natural Policy Gradient based algorithm and prove that it can solve the constrained MARL problem within an error of O(e) with a sample complexity of O(e-6).

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 40 references that are in the map. First posted on arXiv 15 Sep 2022.

Builds on

23 of 40 references.

References · 23

  1. 2017Constrained Policy Optimizationcited by 928
  2. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  3. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  4. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  5. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  6. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  7. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  8. 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
  9. 2019Safe Policies for Reinforcement Learning via Primal-Dual Methodscited by 75
  10. 2018Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learningcited by 61
  11. 2019DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learningcited by 54
  12. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 44
11 more
  1. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 42
  2. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  3. 2019Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learningcited by 33
  4. 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 31
  5. 2019Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methodscited by 24
  6. 2022Achieving Zero Constraint Violation for Constrained Reinforcement Learning via Conservative Natural Policy Gradient Primal-Dual Algorithmcited by 19
  7. 2015Optimal resource allocation for competitive spreading processes on bilayer networkscited by 6
  8. 2022Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction?cited by 6
  9. 2022On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learningcited by 4
  10. 2014Joint Latency and Cost Optimization for Erasure-coded Data Center Storagecited by 2
  11. 2014Iterative Dynamic Water-filling for Fading Multiple-Access Channels with Energy Harvestingcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 3.

5 papers.

  1. 2023Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilitiescited by 15
  2. 2023Safe Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 3
  3. 2025Scalable Ride-Sourcing Vehicle Rebalancing with Service Accessibility Guarantee: A Constrained Mean-Field Reinforcement Learning Approachcited by 0
  4. 2025Multi-Year Maintenance Planning for Large-Scale Infrastructure Systems: A Novel Network Deep Q-Learning Approachcited by 0
  5. 2025Hierarchical Deep Reinforcement Learning Framework for Multi-Year Asset Management Under Budget Constraintscited by 0