mlsanityData to 14 Oct 2026

Major-Minor Mean Field Multi-Agent Reinforcement Learning

arXiv 2303.10665 ↗First posted 19 Mar 2023cs.LG cs.MA math.OCcited by 4

Not on the map. Why?

Abstract (arXiv)

Multi-agent reinforcement learning (MARL) remains difficult to scale to many agents. Recent MARL using Mean Field Control (MFC) provides a tractable and rigorous approach to otherwise difficult cooperative MARL. However, the strict MFC assumption of many independent, weakly-interacting agents is too inflexible in practice. We generalize MFC to instead simultaneously model many similar and few complex agents -- as Major-Minor Mean Field Control (M3FC). Theoretically, we give approximation results for finite agent control, and verify the sufficiency of stationary policies for optimality together with a dynamic programming principle. Algorithmically, we propose Major-Minor Mean Field MARL (M3FMARL) for finite agent systems instead of the limiting system. The algorithm is shown to approximate the policy gradient of the underlying M3FC MDP. Finally, we demonstrate its capabilities experimentally in various scenarios. We observe a strong performance in comparison to state-of-the-art policy gradient MARL methods.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 46 of its 91 references that are in the map. First posted on arXiv 19 Mar 2023.

Builds on

46 of 91 references.

References · 46

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  5. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  6. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  7. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
  8. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 212
  9. 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
  10. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
  11. 2020Fictitious Play for Mean Field Games: Continuous Time Analysis and Applicationscited by 62
  12. 2022Learning in Mean Field Games: A Surveycited by 61
34 more
  1. 2023A Unified Algebraic Perspective on Lipschitz Neural Networkscited by 60
  2. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 59
  3. 2020Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Rewardcited by 50
  4. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 49
  5. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  6. 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 32
  7. 2016Markov-Nash Equilibria in Mean-Field Games with Discounted Costcited by 31
  8. 2021Generalization in Mean Field Games by Learning Master Policiescited by 28
  9. 2021Learning Graphon Mean Field Games and Approximate Nash Equilibriacited by 28
  10. 2020Multi Type Mean Field Reinforcement Learningcited by 26
  11. 2022Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Gamescited by 26
  12. 2019Mean-field Markov decision processes with common noise and open-loop controlscited by 25
  13. 2020A General Framework for Learning Mean-Field Gamescited by 25
  14. 2019Dynamic Programming Principles for Mean-Field Controls with Learningcited by 23
  15. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  16. 2020Partially Observable Mean Field Reinforcement Learningcited by 16
  17. 2021Learning Equilibria in Mean-Field Games: Introducing Mean-Field PSROcited by 14
  18. 2021Decentralized Mean Field Gamescited by 12
  19. 2015Bellman equation and viscosity solutions for mean-field stochastic control problemcited by 11
  20. 2016Mean-Field-Type Games in Engineeringcited by 11
  21. 2022Graphon Mean-Field Control for Cooperative Multi-Agent Reinforcement Learningcited by 11
  22. 2019Massive Autonomous UAV Path Planning: A Neural Network Based Mean-Field Game Theoretic Approachcited by 10
  23. 2021Mean Field Markov Decision Processescited by 10
  24. 2023Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global Statecited by 8
  25. 2020Correlated equilibria and mean field games: a simple modelcited by 7
  26. 2021Discrete-Time Mean Field Control with Environment Statescited by 7
  27. 2021Solving N-player dynamic routing games with congestion: a mean field approachcited by 7
  28. 2020Local Lipschitz Bounds of Deep Neural Networkscited by 5
  29. 2018Optimal Solutions to Infinite-Player Stochastic Teams and Mean-Field Teamscited by 4
  30. 2022Synchronization in a Kuramoto Mean Field Gamecited by 4
  31. 2020Partially Observed Discrete-Time Risk-Sensitive Mean Field Gamescited by 2
  32. 2022Quantitative propagation of chaos for mean field Markov decision process with common noisecited by 2
  33. 2022Correlated equilibria for mean field games with progressive strategiescited by 2
  34. 2022Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systemscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 1; 2026: 2.

4 papers.

  1. 2023Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behaviorcited by 7
  2. 2026Mean-Field Reinforcement Learning without Synchronycited by 1
  3. 2025Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learningcited by 0
  4. 2026Fair Policy Optimization in Major-Minor Weakly Coupled Markov Decision Processescited by 0