mlsanityData to 14 Oct 2026

Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spaces

arXiv 2309.10953 ↗First posted 19 Sep 2023math.OC cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We present the development and analysis of a reinforcement learning (RL) algorithm designed to solve continuous-space mean field game (MFG) and mean field control (MFC) problems in a unified manner. The proposed approach pairs the actor-critic (AC) paradigm with a representation of the mean field distribution via a parameterized score function, which can be efficiently updated in an online fashion, and uses Langevin dynamics to obtain samples from the resulting distribution. The AC agent and the score function are updated iteratively to converge, either to the MFG equilibrium or the MFC optimum for a given mean field problem, depending on the choice of learning rates. A straightforward modification of the algorithm allows us to solve mixed mean field control games (MFCGs). The performance of our algorithm is evaluated using linear-quadratic benchmarks in the asymptotic infinite horizon framework.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 19 of its 44 references that are in the map. First posted on arXiv 19 Sep 2023.

Builds on

19 of 44 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2015Variational Inference with Normalizing Flowscited by 2,170
  5. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 101
  6. 2022q-Learning in Continuous Timecited by 91
  7. 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 84
  8. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
  9. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 59
  10. 2020Entropy Regularization for Mean Field Games with Learningcited by 58
  11. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 49
  12. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 38
7 more
  1. 2021Mean Field Games Flock! The Reinforcement Learning Waycited by 31
  2. 2023Actor-Critic learning for mean-field control in continuous timecited by 22
  3. 2021Deep Learning for Mean Field Games and Mean Field Control with Applications to Financecited by 20
  4. 2021Reinforcement Learning for Mean Field Games, with Applications to Economicscited by 15
  5. 2022Global Convergence of Two-timescale Actor-Critic for Solving Linear Quadratic Regulatorcited by 8
  6. 2022Reinforcement Learning Algorithm for Mixed Mean Field Control Gamescited by 7
  7. 2022Reinforcement Learning for Intra-and-Inter-Bank Borrowing and Lending Mean Field Control Gamecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 2.

6 papers.

  1. 2024Unified continuous-time q-learning for mean-field game and mean-field control problemscited by 13
  2. 2026Model-free policy gradient for discrete-time mean-field controlcited by 4
  3. 2026Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithmscited by 4
  4. 2025Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spacescited by 3
  5. 2024Reinforcement Learning for Finite Space Mean-Field Type Gamescited by 2
  6. 2025Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamicscited by 2