mlsanityData to 7 Oct 2026

Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learning

arXiv 2006.11917 ↗First posted 21 Jun 2020cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

Multi-agent reinforcement learning (MARL) achieves significant empirical successes. However, MARL suffers from the curse of many agents. In this paper, we exploit the symmetry of agents in MARL. In the most generic form, we study a mean-field MARL problem. Such a mean-field MARL is defined on mean-field states, which are distributions that are supported on continuous space. Based on the mean embedding of the distributions, we propose MF-FQI algorithm that solves the mean-field MARL and establishes a non-asymptotic analysis for MF-FQI algorithm. We highlight that MF-FQI algorithm enjoys a "blessing of many agents" property in the sense that a larger number of observed agents improves the performance of MF-FQI algorithm.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 8; 2024: 2; 2025: 1; 2026: 2.

19 papers.

Sort:

By area:

  1. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  2. 2023Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithmscited by 25
  3. 2021Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning: A Principled Approachcited by 13
  4. 2022Minimax Optimal Kernel Operator Learning via Multilevel Trainingcited by 13
  5. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  6. 2022Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARLcited by 12
  7. 2023Dealing With Non-stationarity in Decentralized Cooperative Multi-Agent Deep Reinforcement Learning via Multi-Timescale Learningcited by 12
  8. 2023Transformers in Reinforcement Learning: A Surveycited by 11
  9. 2022Indexability is Not Enough for Whittle: Improved, Near-Optimal Algorithms for Restless Banditscited by 10
  10. 2023Learning Regularized Monotone Graphon Mean-Field Gamescited by 8
  11. 2025LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systemscited by 8
  12. 2024Graphon Mean Field Games with a Representative Player: Analysis and Learning Algorithmcited by 5
7 more
  1. 2023Population-size-Aware Policy Optimization for Mean-Field Gamescited by 3
  2. 2023Safe Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 3
  3. 2024Exploiting Approximate Symmetry for Efficient Multi-Agent Reinforcement Learningcited by 2
  4. 2023Networked Communication for Decentralised Agents in Mean-Field Gamescited by 0
  5. 2023Amortized Network Intervention to Steer the Excitatory Point Processescited by 0
  6. 2026Explicit Credit Assignment through Local Rewards and Dependence Graphs in Multi-Agent Reinforcement Learningcited by 0
  7. 2026Towards Scaling Reinforcement Learning to Massive Populations: Learning Mean-Field Representationscited by 0