mlsanityData to 7 Oct 2026

Robust Q-learning for mean-field control under Wasserstein uncertainty in common noise

arXiv 2606.20356 ↗First posted 18 Jun 2026math.OC cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this article, we present a robust Q-learning algorithm for discrete-time mean-field control problems under Wasserstein uncertainty in the common noise law. The algorithm combines a quantization-and-projection scheme with a Wasserstein dual reformulation on the common-noise space. We establish its convergence together with finite-time iteration bounds for both synchronous and asynchronous learning schemes. Numerical experiments on systemic risk and epidemic models compare the asynchronous implementation with an idealized Bellman iteration, illustrate the robustness-performance tradeoff under common-noise misspecification, and report the observed convergence behavior of the asynchronous Q-learning algorithm.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 53 of its 91 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

53 of 91 references.

References · 53

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  3. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  4. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 275
  5. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  6. 2016Distributionally Robust Stochastic Optimization with Wasserstein Distancecited by 182
  7. 2020Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Modelcited by 108
  8. 2022Robust Reinforcement Learning using Offline Datacited by 95
  9. 2020Finite-Time Analysis of Asynchronous Stochastic Approximation and Q-Learningcited by 93
  10. 2020Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reductioncited by 93
  11. 2022Settling the Sample Complexity of Model-Based Offline Reinforcement Learningcited by 82
  12. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 65
38 more
  1. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 56
  2. 2022Learning in Mean Field Games: A Surveycited by 56
  3. 2017Reinforcement Learning under Model Mismatchcited by 52
  4. 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coveragecited by 51
  5. 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 45
  6. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 44
  7. 2020Q-Learning in Regularized Mean-field Gamescited by 40
  8. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 33
  9. 2019Mean-field Markov decision processes with common noise and open-loop controlscited by 24
  10. 2019Dynamic Programming Principles for Mean-Field Controls with Learningcited by 22
  11. 2021Numerical Methods for Mean Field Games and Mean Field Type Controlcited by 21
  12. 2023Actor-Critic learning for mean-field control in continuous timecited by 19
  13. 2023Recent Developments in Machine Learning Methods for Stochastic Control and Gamescited by 19
  14. 2023Continuous-time q-learning for mean-field control problemscited by 17
  15. 2023Policy Gradient Algorithms for Robust MDPs with Non-Rectangular Uncertainty Setscited by 15
  16. 2020Distributionally Robust Markov Decision Processes and their Connection to Risk Measurescited by 12
  17. 2016Dynamic programming for optimal control of stochastic McKean-Vlasov dynamicscited by 11
  18. 2020Exploratory LQG Mean Field Games with Entropy Regularizationcited by 11
  19. 2017Computational aspects of robust optimized certainty equivalents and option pricingcited by 7
  20. 2024Unified continuous-time q-learning for mean-field game and mean-field control problemscited by 7
  21. 2019McKean-Vlasov optimal control: the dynamic programming principlecited by 6
  22. 2020Linear-Quadratic Zero-Sum Mean-Field Type Games: Optimality Conditions and Policy Optimizationcited by 5
  23. 2026Continuous-time q-learning for mean-field control with common noise, part-I: Theoretical foundationscited by 5
  24. 2015Discrete time McKean-Vlasov control problem: a dynamic programming approachcited by 4
  25. 2018Optimal Solutions to Infinite-Player Stochastic Teams and Mean-Field Teamscited by 4
  26. 2020Policy Optimization for Linear-Quadratic Zero-Sum Mean-Field Type Gamescited by 4
  27. 2024Analysis of Multiscale Reinforcement Q-Learning Algorithms for Mean Field Control Gamescited by 3
  28. 2024Robust Cooperative Multi-Agent Reinforcement Learning:A Mean-Field Type Game Perspectivecited by 3
  29. 2026Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithmscited by 3
  30. 2017Robust Mean Field Linear-Quadratic-Gaussian Games with Unknown L2-Disturbancecited by 2
  31. 2020McKean-Vlasov optimal control: limit theory and equivalence between different formulationscited by 2
  32. 2020Extended mean field control problem: a propagation of chaos resultcited by 2
  33. 2022Quantitative propagation of chaos for mean field Markov decision process with common noisecited by 2
  34. 2024Q-Learning under Finite Model Uncertaintycited by 2
  35. 2024Markov-Nash equilibria in mean-field games under model uncertaintycited by 2
  36. 2025Robust mean-field control under common noise uncertaintycited by 2
  37. 2026Mean-Field Games Under Model Uncertaintycited by 2
  38. 2019Social Optima in Robust Mean Field LQG Control: From Finite to Infinite Horizoncited by 1

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.