mlsanityData to 7 Oct 2026

Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction?

arXiv 2203.00035 ↗First posted 28 Feb 2022cs.LG cs.MAcited by 6

Not on the map. Why?

Abstract (arXiv)

Mean-Field Control (MFC) is a powerful tool to solve Multi-Agent Reinforcement Learning (MARL) problems. Recent studies have shown that MFC can well-approximate MARL when the population size is large and the agents are exchangeable. Unfortunately, the presumption of exchangeability implies that all agents uniformly interact with one another which is not true in many practical scenarios. In this article, we relax the assumption of exchangeability and model the interaction between agents via an arbitrary doubly stochastic matrix. As a result, in our framework, the mean-field `seen' by different agents are different. We prove that, if the reward of each agent is an affine function of the mean-field seen by that agent, then one can approximate such a non-uniform MARL problem via its associated MFC problem within an error of e=O(frac1√N[√|X| + √|U|]) where N is the population size and |X|, |U| are the sizes of state and action spaces respectively. Finally, we develop a Natural Policy Gradient (NPG) algorithm that can provide a solution to the non-uniform MARL with an error O(maxe,ε) and a sample complexity of O(ε-3) for any ε >0.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 13 of its 29 references that are in the map. First posted on arXiv 28 Feb 2022.

Builds on

13 of 29 references.

References · 13

  1. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  2. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  3. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  4. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  5. 2019DeepPool: Distributed Model-free Algorithm for Ride-sharing using Deep Reinforcement Learningcited by 54
  6. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 44
  7. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 42
  8. 2018Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimizationcited by 40
  9. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  10. 2019Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learningcited by 33
  11. 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 31
  12. 2021Learning Graphon Mean Field Games and Approximate Nash Equilibriacited by 28
1 more
  1. 2015Optimal resource allocation for competitive spreading processes on bilayer networkscited by 6

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 2.

6 papers.

  1. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  2. 2023Convergence Rates for Localized Actor-Critic in Networked Markov Potential Gamescited by 9
  3. 2023Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global Statecited by 8
  4. 2022Mean-Field Approximation of Cooperative Constrained Multi-Agent Reinforcement Learning (CMARL)cited by 5
  5. 2022On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learningcited by 4
  6. 2022Hypergraphon Mean Field Gamescited by 3