# Graphon Mean-Field Control for Cooperative Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2209.04808 · HTML: https://mlsanity.com/paper/2209.04808/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2022.

## Built on by (most cited first, 11 of 11)

- [Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2412.00661/) (2024)
- [Efficient Reinforcement Learning for Global Decision Making in the Presence of Local Agents at Scale](https://mlsanity.com/paper/2403.00222/) (2024)
- [Major-Minor Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.10665/) (2023)
- [Learning Mean Field Control on Sparse Graphs](https://mlsanity.com/paper/2501.17079/) (2025)
- [Learning Approximate Nash Equilibria in Cooperative Multi-Agent Reinforcement Learning via Mean-Field Subsampling](https://mlsanity.com/paper/2603.03759/) (2026)
- [Factored Value Functions for Graph-Based Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2601.11401/) (2026)
- [Graphon Mean-Field Subsampling for Cooperative Heterogeneous Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2602.16196/) (2026)
- [Sparse Mean Field Load Balancing in Large Localized Queueing Systems](https://mlsanity.com/paper/2312.12973/) (2023)
- [Scalable Ride-Sourcing Vehicle Rebalancing with Service Accessibility Guarantee: A Constrained Mean-Field Reinforcement Learning Approach](https://mlsanity.com/paper/2503.24183/) (2025)
- [Mean-Field Control on Sparse Graphs: From Local Limits to GNNs via Neighborhood Distributions](https://mlsanity.com/paper/2601.21477/) (2026)
- [Decentralized Strategies for Finite Population LQG Social Control: A Reinforcement Learning Approach](https://mlsanity.com/paper/2608.30244/) (2026)

## Abstract (arXiv)

The marriage between mean-field theory and reinforcement learning has shown a great capacity to solve large-scale control problems with homogeneous agents. To break the homogeneity restriction of mean-field theory, a recent interest is to introduce graphon theory to the mean-field paradigm. In this paper, we propose a graphon mean-field control (GMFC) framework to approximate cooperative multi-agent reinforcement learning (MARL) with nonuniform interactions and show that the approximate order is of O(frac1√N), with N the number of agents. By discretizing the graphon index of GMFC, we further introduce a smaller class of GMFC called block GMFC, which is shown to well approximate cooperative MARL. Our empirical studies on several examples demonstrate that our GMFC approach is comparable with the state-of-art MARL algorithms while enjoying better scalability.