mlsanityData to 7 Oct 2026

Actor-Critic learning for mean-field control in continuous time

arXiv 2303.06993 ↗First posted 13 Mar 2023stat.ML math.OCcited by 19

Not on the map. Why?

Abstract (arXiv)

We study policy gradient for mean-field control in continuous time in a reinforcement learning setting. By considering randomised policies with entropy regularisation, we derive a gradient expectation representation of the value function, which is amenable to actor-critic type algorithms, where the value functions and the policies are learnt alternately based on observation samples of the state and model-free estimation of the population state distribution, either by offline or online learning. In the linear-quadratic mean-field framework, we obtain an exact parametrisation of the actor and critic functions defined on the Wasserstein space. Finally, we illustrate the results of our algorithms with some numerical experiments on concrete examples.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 19 of its 34 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

19 of 34 references.

References · 19

  1. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 76
  2. 2022q-Learning in Continuous Timecited by 68
  3. 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 62
  4. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 56
  5. 2020Entropy Regularization for Mean Field Games with Learningcited by 50
  6. 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 45
  7. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 44
  8. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 42
  9. 2021Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approachcited by 27
  10. 2019Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methodscited by 24
  11. 2019Numerical resolution of McKean-Vlasov FBSDEs using neural networkscited by 19
  12. 2021Reinforcement Learning for Mean Field Games, with Applications to Economicscited by 14
7 more
  1. 2022Learning High-Dimensional McKean-Vlasov Forward-Backward Stochastic Differential Equations with General Distribution Dependencecited by 13
  2. 2016Dynamic programming for optimal control of stochastic McKean-Vlasov dynamicscited by 11
  3. 2021DeepSets and their derivative networks for solving symmetric PDEscited by 10
  4. 2021A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problemscited by 10
  5. 2022Mean-field neural networks-based algorithms for McKean-Vlasov control problems *cited by 9
  6. 2020Ito's formula for flows of measures on semimartingalescited by 6
  7. 2014A Probabilistic approach to classical solutions of the master equation for large population equilibriacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 3; 2026: 7.

19 papers.

Sort:

  1. 2023Continuous-time q-learning for mean-field control problemscited by 17
  2. 2024Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problemscited by 10
  3. 2025Accuracy of Discretely Sampled Stochastic Policies in Continuous-time Reinforcement Learningcited by 9
  4. 2024Unified continuous-time q-learning for mean-field game and mean-field control problemscited by 7
  5. 2024Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problemscited by 7
  6. 2023Actor critic learning algorithms for mean-field control with moment neural networkscited by 6
  7. 2023Deep Reinforcement Learning for Infinite Horizon Mean Field Problems in Continuous Spacescited by 6
  8. 2024Continuous-time Risk-sensitive Reinforcement Learning via Quadratic Variation Penaltycited by 5
  9. 2026Continuous-time q-learning for mean-field control with common noise, part-I: Theoretical foundationscited by 5
  10. 2024Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noisecited by 4
  11. 2024Analysis of Multiscale Reinforcement Q-Learning Algorithms for Mean Field Control Gamescited by 3
  12. 2025Convergence of Actor-Critic Learning for Mean Field Games and Mean Field Control in Continuous Spacescited by 3
7 more
  1. 2026Model-free policy gradient for discrete-time mean-field controlcited by 3
  2. 2026Continuous-time q-learning for mean-field control with common noise, part-II: q-learning algorithmscited by 3
  3. 2026Actor-Critic Learning for Extended Mean Field Control with Deterministic Policiescited by 1
  4. 2025Neural Mean-Field Games: Extending Mean-Field Game Theory with Neural Stochastic Differential Equationscited by 0
  5. 2026Robust Q-learning for mean-field control under Wasserstein uncertainty in common noisecited by 0
  6. 2026Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Datacited by 0
  7. 2026Entropy Regularization Improves Policy Robustness in Continuous-Time Reinforcement Learningcited by 0