mlsanityData to 7 Oct 2026

A Policy Gradient Framework for Stochastic Optimal Control Problems with Global Convergence Guarantee

arXiv 2302.05816 ↗First posted 11 Feb 2023math.OC cs.LG cs.SYcited by 6

Not on the map. Why?

Abstract (arXiv)

We consider policy gradient methods for stochastic optimal control problem in continuous time. In particular, we analyze the gradient flow for the control, viewed as a continuous time limit of the policy gradient method. We prove the global convergence of the gradient flow and establish a convergence rate under some regularity assumptions. The main novelty in the analysis is the notion of local optimal control function, which is introduced to characterize the local optimality of the iterate.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 22 of its 55 references that are in the map. First posted on arXiv 11 Feb 2023.

Builds on

22 of 55 references.

References · 22

  1. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  2. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  3. 2020A Finite Time Analysis of Two Time-Scale Actor Critic Methodscited by 120
  4. 2019A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problemscited by 95
  5. 2020Entropy Regularization for Mean Field Games with Learningcited by 50
  6. 2020Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approachcited by 35
  7. 2019Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: II -- The Finite Horizon Casecited by 34
  8. 2020A mean-field analysis of two-player zero-sum gamescited by 28
  9. 2021Exploratory HJB equations and their convergencecited by 28
  10. 2021Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networkscited by 27
  11. 2019Deep Learning Methods for Mean Field Control Problems with Delaycited by 24
  12. 2021A Neural Network Approach for High-Dimensional Optimal Control Applied to Multi-Agent Path Findingcited by 18
10 more
  1. 2020Global Convergence of Policy Gradient for Linear-Quadratic Mean-Field Control/Game in Continuous Timecited by 15
  2. 2021Signatured Deep Fictitious Play for Mean Field Games with Common Noisecited by 12
  3. 2020Exploratory LQG Mean Field Games with Entropy Regularizationcited by 11
  4. 2020Gradient Flows for Regularized Stochastic Control Problemscited by 8
  5. 2020A modified MSA for stochastic control problemscited by 8
  6. 2019A neural network based policy iteration algorithm with global H2-superlinear convergence for stochastic games on domainscited by 7
  7. 2020Solving stochastic optimal control problem via stochastic maximum principle with deep learning methodcited by 6
  8. 2021Mean field control problems for vaccine distributioncited by 4
  9. 2021Recurrent Neural Networks for Stochastic Control Problems with Delaycited by 2
  10. 2022The Modified MSA, a Gradient Flow and Convergencecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergencecited by 18
  2. 2024Entropy annealing for policy mirror descent in continuous time and spacecited by 9
  3. 2024Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problemscited by 7
  4. 2025Neural Actor-Critic Methods for Hamilton-Jacobi-Bellman PDEs: Asymptotic Analysis and Numerical Studiescited by 3
  5. 2025Continuous Policy and Value Iteration for Relaxed Stochastic Control Problems and Its Convergencecited by 1
  6. 2026Policy Gradient for Continuous-Time Robust Markov Decision Processescited by 0