Solving Time-Continuous Stochastic Optimal Control Problems: Algorithm Design and Convergence Analysis of Actor-Critic Flow
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
We propose an actor-critic framework to solve the time-continuous stochastic optimal control problem. A least square temporal difference method is applied to compute the value function for the critic. The policy gradient method is implemented as policy improvement for the actor. Our key contribution lies in establishing a linear rate of convergence for our proposed actor-critic flow. Theoretical findings are further validated through numerical examples, showing the efficacy of our approach in practical applications.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. First posted on arXiv 27 Feb 2024.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2024: 3; 2025: 1; 2026: 4.
8 papers.
- 2024Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergencecited by 19
- 2024Entropy annealing for policy mirror descent in continuous time and spacecited by 12
- 2024Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problemscited by 7
- 2026Rare Event Analysis via Stochastic Optimal Controlcited by 4
- 2025Continuous Policy and Value Iteration for Relaxed Stochastic Control Problems and Its Convergencecited by 1
- 2026Dual Approaches to Stochastic Control via SPDEs and the Pathwise Hopf Formulacited by 0
- 2026Self-supervised In-context Operator Learning for Stochastic Mean-Field Controlcited by 0
- 2026Amortized Score-Hamiltonian Policy Iteration: A Grid-Free Scheme for Relaxed Stochastic Control Problemscited by 0
- 2026Amortized Score-Hamiltonian Policy Iteration: A Grid-Free Scheme for Relaxed Stochastic Control Problems3 Oct 2026
- 2026Self-supervised In-context Operator Learning for Stochastic Mean-Field Control18 Aug 2026
- 2026Rare Event Analysis via Stochastic Optimal Control14 Apr 2026
- 2026Dual Approaches to Stochastic Control via SPDEs and the Pathwise Hopf Formula9 Apr 2026
- 2025Continuous Policy and Value Iteration for Relaxed Stochastic Control Problems and Its Convergence9 Jun 2025
- 2024Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergence24 Dec 2024
- 2024Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problems24 Sep 2024
- 2024Entropy annealing for policy mirror descent in continuous time and space30 May 2024
Similar, not cited
Close in wording, no citation either way
- 2024Compatible Gradient Approximations for Actor-Critic Algorithms
- 2025Global Optimality of Single-Timescale Actor-Critic under Continuous State-Action Space: A Study on Linear Quadratic Regulator
- 2024On the Convergence of Single-Timescale Actor-Critic
- 2026An Actor-Critic Framework for Continuous-Time Jump-Diffusion Controls with Normalizing Flows
- 2025Convergence of an actor-critic gradient flow for entropy regularised MDPs in general spaces
- 2025Quasi-Newton Compatible Actor-Critic for Deterministic Policies
- 2024Control randomisation approach for policy gradient and application to reinforcement learning in optimal switching
- 2025Finite Time Analysis of Constrained Natural Critic-Actor Algorithm with Improved Sample Complexity