mlsanityData to 14 Oct 2026

Stationary Robust Mean-Field Games under Model Mismatches

arXiv 2606.22579 ↗First posted 21 Jun 2026cs.LG cs.GTcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Deploying multi-agent reinforcement learning (MARL) in the real world is often limited by model mismatches between the training simulators and the true environment, which could be further amplified through strategic interactions and result in severe performance degradation upon deployment. Distributional robustness offers a principled response by optimizing policies against worst-case transition models drawn from an uncertainty set, but standard robust MARL frameworks become increasingly intractable as the number of agents grows. This paper develops an infinite-horizon, stationary mean-field game framework that incorporates distributional model uncertainty directly into the population-coupled dynamics. We establish a robust dynamic programming principle with a contractive Bellman operator and prove the existence of a stationary robust mean-field equilibrium via a fixed-point argument. We further develop the first concrete algorithm with convergence guarantees. We then connect the mean-field solution to a finite-population robust game whose ambiguity sets depend on the empirical distribution, showing that the mean-field equilibrium policy induces approximate equilibrium behavior as the population size increases. Under a contractive robust-dynamics regime, we further obtain explicit non-asymptotic error bounds. Numerical experiments further illustrate the qualitative and quantitative impact of robustness under multiple uncertainty models, validating our theoretical findings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 60 of its 109 references that are in the map. First posted on arXiv 21 Jun 2026.

Builds on

60 of 109 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Mean-Field Reinforcement Learning without Synchrony20 Feb
  2. 2025Robust Mean Field Social Control: A Unified Reinforcement Learning Framework27 Feb
  3. 2024When is Mean-Field Reinforcement Learning Tractable and Relevant?8 Feb

Other references · 57

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  2. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  3. 2017Robust Adversarial Reinforcement Learningcited by 451
  4. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  5. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  6. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 138
  7. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  8. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 96
  9. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 93
  10. 2022Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexitycited by 87
  11. 2021Sample Complexity of Robust Reinforcement Learning with a Generative Modelcited by 83
  12. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
38 more
  1. 2020Robust Reinforcement Learning using Adversarial Populationscited by 66
  2. 2020Fictitious Play for Mean Field Games: Continuous Time Analysis and Applicationscited by 62
  3. 2022Learning in Mean Field Games: A Surveycited by 61
  4. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpointcited by 56
  5. 2019Wasserstein Robust Reinforcement Learningcited by 54
  6. 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 52
  7. 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coveragecited by 51
  8. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 49
  9. 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 48
  10. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  11. 2020Q-Learning in Regularized Mean-field Gamescited by 42
  12. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  13. 2015Learning in Mean Field Games: the Fictitious Playcited by 38
  14. 2023A Finite Sample Complexity Bound for Distributionally Robust Q-learningcited by 36
  15. 2016Markov-Nash Equilibria in Mean-Field Games with Discounted Costcited by 31
  16. 2022Policy Mirror Ascent for Efficient and Independent Learning in Mean Field Gamescited by 26
  17. 2024Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithmscited by 26
  18. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 25
  19. 2021Numerical Methods for Mean Field Games and Mean Field Type Controlcited by 22
  20. 2023Robust Average-Reward Markov Decision Processescited by 22
  21. 2018Decentralised Learning in Systems with Many, Many Strategic Agentscited by 20
  22. 2023Model-Free Robust Average-Reward Reinforcement Learningcited by 20
  23. 2023Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospectscited by 18
  24. 2024Sample Complexity of Offline Distributionally Robust Linear Markov Decision Processescited by 14
  25. 2018Finite mean field games: fictitious play and convergence to a first order continuous mean field gamecited by 9
  26. 2020A case study on stochastic games on large graphs in mean field and sparse regimescited by 9
  27. 2020Optimal incentives to mitigate epidemics: a Stackelberg mean field game approachcited by 9
  28. 2024Model-Free Robust Reinforcement Learning with Sample Complexity Analysiscited by 9
  29. 2017Approximate Nash Equilibria in Partially Observed Stochastic Games with Mean-Field Interactionscited by 8
  30. 2025Sample Complexity of Distributionally Robust Average-Reward Reinforcement Learningcited by 7
  31. 2025Sample Complexity of Distributionally Robust Off-Dynamics Reinforcement Learning with Online Interactioncited by 7
  32. 2024Breaking the Curse of Multiagency in Robust Multi-Agent Reinforcement Learningcited by 6
  33. 2025ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learningcited by 6
  34. 2024Minimax-Optimal Multi-Agent Robust Reinforcement Learningcited by 5
  35. 2025Bellman Optimality of Average-Reward Robust Markov Decision Processes with a Constant Gaincited by 5
  36. 2024Robust Cooperative Multi-Agent Reinforcement Learning:A Mean-Field Type Game Perspectivecited by 4
  37. 2021Exploration noise for learning linear-quadratic mean field gamescited by 3
  38. 2025Sample-Efficient Distributionally Robust Multi-Agent Reinforcement Learning via Online Interactioncited by 3

50 most cited of 57 All 57 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Robust Mean-Field Games with Risk Aversion and Bounded Rationality
  2. 2026Taming the Curses of Multiagency in Robust Markov Games with Large State Space through Linear Function Approximation
  3. 2025Computing and Learning Stationary Mean Field Equilibria with Low-Dimensional Interactions: Algorithms and Applications ↗
  4. 2026Interactive Distributionally Robust Multi-Agent Learning with General Function Approximation
  5. 2026Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games
  6. 2026Strategically Robust Multi-Agent Reinforcement Learning with Linear Function Approximation
  7. 2025Distributionally Robust Online Markov Game with Linear Function Approximation
  8. 2026Mean Field Reinforcement Learning