mlsanityData to 14 Oct 2026

Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning

arXiv 2306.17052 ↗First posted 29 Jun 2023cs.LG cs.AI cs.MAcited by 4

Not on the map. Why?

Abstract (arXiv)

Many applications, e.g., in shared mobility, require coordinating a large number of agents. Mean-field reinforcement learning addresses the resulting scalability challenge by optimizing the policy of a representative agent interacting with the infinite population of identical agents instead of considering individual pairwise interactions. In this paper, we address an important generalization where there exist global constraints on the distribution of agents (e.g., requiring capacity constraints or minimum coverage requirements to be met). We propose Safe-M3-UCRL, the first model-based mean-field reinforcement learning algorithm that attains safe policies even in the case of unknown transitions. As a key ingredient, it uses epistemic uncertainty in the transition model within a log-barrier approach to ensure pessimistic constraints satisfaction with high probability. Beyond the synthetic swarm motion benchmark, we showcase Safe-M3-UCRL on the vehicle repositioning problem faced by many shared mobility operators and evaluate its performance through simulations built on vehicle trajectory data from a service provider in Shenzhen. Our algorithm effectively meets the demand in critical areas while ensuring service accessibility in regions with low demand.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 36 of its 76 references that are in the map. First posted on arXiv 29 Jun 2023.

Builds on

36 of 76 references.

References · 36

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  6. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  7. 2017Safe Reinforcement Learning via Shieldingcited by 449
  8. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  9. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  10. 2018Safe Exploration in Continuous Action Spacescited by 247
  11. 2017Parameter Space Noise for Explorationcited by 161
  12. 2020Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planningcited by 86
24 more
  1. 2022Learning in Mean Field Games: A Surveycited by 61
  2. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 59
  3. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 49
  4. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 49
  5. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  6. 2019On Online Learning in Kernelized Markov Decision Processescited by 35
  7. 2021Multi-Agent Constrained Policy Optimisationcited by 35
  8. 2019Calibrated Model-Based Deep Reinforcement Learningcited by 31
  9. 2019Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methodscited by 28
  10. 2014Control of Robotic Mobility-On-Demand Systems: a Queueing-Theoretical Perspectivecited by 25
  11. 2019Mean-field Markov decision processes with common noise and open-loop controlscited by 25
  12. 2017Data-Driven Model Predictive Control of Autonomous Mobility-on-Demand Systemscited by 23
  13. 2019Dynamic Programming Principles for Mean-Field Controls with Learningcited by 23
  14. 2020Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learningcited by 21
  15. 2020Global Convergence of Policy Gradient for Linear-Quadratic Mean-Field Control/Game in Continuous Timecited by 20
  16. 2021Reinforcement Learning for Mean Field Games, with Applications to Economicscited by 15
  17. 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14
  18. 2020Multi-Agent Safe Planning with Gaussian Processescited by 13
  19. 2016Limit theory for controlled McKean-Vlasov dynamicscited by 12
  20. 2021Mean Field Markov Decision Processescited by 10
  21. 2020Linear-Quadratic Zero-Sum Mean-Field Type Games: Optimality Conditions and Policy Optimizationcited by 6
  22. 2020Multi-Agent Reinforcement Learning for Unmanned Aerial Vehicle Coordination by Multi-Critic Policy Gradient Optimizationcited by 5
  23. 2021Safe Deep Reinforcement Learning for Multi-Agent Systems with Continuous Action Spacescited by 5
  24. 2022Mean-Field Approximation of Cooperative Constrained Multi-Agent Reinforcement Learning (CMARL)cited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024Scalable Offline Reinforcement Learning for Mean Field Gamescited by 1
  2. 2025Scalable Ride-Sourcing Vehicle Rebalancing with Service Accessibility Guarantee: A Constrained Mean-Field Reinforcement Learning Approachcited by 0
  3. 2026Fair Policy Optimization in Major-Minor Weakly Coupled Markov Decision Processescited by 0
  4. 2026Mean field games as a tool for AI safety: a worked example from the July 2026 Hugging Face incidentcited by 0