Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
Not on the map. Why?
Abstract (arXiv)
We study continuous-time and possibly high-dimensional stochastic control problems where drift coefficients and running reward functions are unknown. Due to these missing model primitives, we take the exploratory, reinforcement learning (RL) framework of Wang, Zariphopoulou, and Zhou(2020) with relaxed controls and entropy regularization. The objective is to develop theoretically grounded, efficient and scalable RL algorithms to learn both the optimal value functions (which also solve the exploratory HJB equation) and optimal exploratory feedback control policies. When the diffusion coefficients do not contain control, we employ probabilistic representations of both the optimal value function and its gradient based on an auxiliary state process depending only on the diffusion part of the original dynamics. With a delicate analysis on some properly defined mappings and their fixed points, this leads to the introduction of our policy iteration algorithms and their convergence. We demonstrate the performance of our algorithms through various numerical examples. Finally, we study a special control-dependent diffusion case where probability representation of the Hessian is called for.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 14 Sep 2026.
Builds on
26 of 40 references.
References · 26
- 2017Solving high-dimensional partial differential equations using deep learningcited by 549
- 2021Recent Advances in Reinforcement Learning in Financecited by 102
- 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 76
- 2022q-Learning in Continuous Timecited by 68
- 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 62
- 2021Exploratory HJB equations and their convergencecited by 28
- 2022Convergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problemscited by 19
- 2023Recent Developments in Machine Learning Methods for Stochastic Control and Gamescited by 19
- 2023Continuous-time q-learning for mean-field control problemscited by 17
- 2024Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problemscited by 10
- 2024Regret of exploratory policy improvement and q-learningcited by 10
- 2026Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approachcited by 10
14 more
- 2024Entropy annealing for policy mirror descent in continuous time and spacecited by 9
- 2024Policy Iteration for Exploratory Hamilton--Jacobi--Bellman Equationscited by 9
- 2024Mean--Variance Portfolio Selection by Continuous-Time Reinforcement Learning: Algorithms, Regret Analysis, and Empirical Studycited by 7
- 2025Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and Statecited by 7
- 2022Convergence of Policy Iteration for Entropy-Regularized Stochastic Control Problemscited by 6
- 2024Exploratory Optimal Stopping: A Singular Control Formulationcited by 5
- 2024Convergence Analysis for Entropy-Regularized Control Problems: A Probabilistic Approachcited by 4
- 2026Policy Gradient for Continuous-Time Mean-Field Controlcited by 4
- 2025A Two-fold Randomization Framework for Impulse Control Problemscited by 3
- 2026Deterministic Policy Gradient for Learning Equilibrium in Time-Inconsistent Control Problemscited by 3
- 2025Data-Driven Exploration for a Class of Continuous-Time Indefinite Linear--Quadratic Reinforcement Learning Problemscited by 2
- 2026Policy Iteration Achieves Regularized Equilibrium under Time Inconsistencycited by 2
- 2026Discretization error from regularized Reinforcement Learning to continuous-time stochastic controlcited by 2
- 2026A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficientscited by 2
Built on by
Not cited yet.