Graphon Mean-Field Subsampling for Cooperative Heterogeneous Multi-Agent Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Coordinating large populations of interacting agents is a central challenge in multi-agent reinforcement learning (MARL), where the size of the joint state-action space scales exponentially with the number of agents. Mean-field methods alleviate this burden by aggregating agent interactions, but these approaches assume homogeneous interactions. Recent graphon-based frameworks capture heterogeneity, but are computationally expensive as the number of agents grows. Therefore, we introduce textttGMFS, a Graphon Mean-Field Subsampling framework for scalable cooperative MARL with heterogeneous agent interactions. By subsampling kappa agents according to interaction strength, we approximate the graphon-weighted mean-field and learn a policy with sample complexity poly(kappa) and optimality gap O(1/√kappa). We verify our theory with numerical simulations in robotic coordination, showing that textttGMFS achieves near-optimal performance.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 56 of its 86 references that are in the map. First posted on arXiv 18 Feb 2026.
Builds on
56 of 86 references.
References · 56
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
- 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
- 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
- 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 224
- 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
- 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
- 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
- 2019Learning with Good Feature Representations in Bandits and in RL with a Generative Modelcited by 124
- 2019Variance-reduced Q-learning is minimax optimalcited by 72
38 more
- 2018Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertisingcited by 65
- 2019Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learningcited by 57
- 2021A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variantscited by 57
- 2017Variance Reduced Value Iteration and Faster Algorithms for Solving Markov Decision Processescited by 56
- 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 56
- 2022Learning in Mean Field Games: A Surveycited by 56
- 2018Multi-Agent Reinforcement Learning Based Resource Allocation for UAV Networkscited by 54
- 2020Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Rewardcited by 46
- 2013Stochastic blockmodel approximation of a graphon: Theory and consistent estimationcited by 38
- 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
- 2014A Consistent Histogram Estimator for Exchangeable Graph Modelscited by 36
- 2020Active Sampling for Min-Max Fairnesscited by 33
- 2022On the Complexity of Adversarial Decision Makingcited by 32
- 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 31
- 2021Learning Graphon Mean Field Games and Approximate Nash Equilibriacited by 28
- 2021Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approachcited by 27
- 2023Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noisecited by 26
- 2018Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learningcited by 24
- 2018Near-Optimal Time and Sample Complexities for Solving Discounted Markov Decision Process with a Generative Modelcited by 22
- 2022More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorizationcited by 19
- 2021Finite-Sample Analysis of Off-Policy TD-Learning via Generalized Bellman Operatorscited by 17
- 2022Online Adaptive Policy Selection in Time-Varying Systems: No-Regret via Contractive Perturbationscited by 14
- 2021Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning: A Principled Approachcited by 13
- 2020Graphon Mean Field Games and the GMFG Equationscited by 11
- 2022Graphon Mean-Field Control for Cooperative Multi-Agent Reinforcement Learningcited by 10
- 2023Provable Representation with Efficient Planning for Partial Observable Reinforcement Learningcited by 10
- 2023Cooperative Multi-Agent Reinforcement Learning: Asynchronous Communication and Linear Function Approximationcited by 9
- 2024Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learningcited by 9
- 2024Efficient Reinforcement Learning for Global Decision Making in the Presence of Local Agents at Scalecited by 8
- 2024Online Policy Optimization in Unknown Nonlinear Systemscited by 8
- 2024Peer-to-Peer Learning Dynamics of Wide Neural Networkscited by 8
- 2025Multi-Agent Reinforcement Learning in Wireless Distributed Networks for 6Gcited by 7
- 2023Efficient Planning in Combinatorial Action Spaces with Applications to Cooperative Multi-Agent Reinforcement Learningcited by 6
- 2024Truncated Variance Reduced Value Iterationcited by 6
- 2024A Local Information Aggregation based Multi-Agent Reinforcement Learning for Robot Swarm Dynamic Task Allocationcited by 6
- 2023Learning Regularized Graphon Mean-Field Games with Unknown Graphonscited by 4
- 2025The Structural Complexity of Matrix-Vector Multiplicationcited by 4
- 2025Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdowncited by 4
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.