Dual Approaches to Stochastic Control via SPDEs and the Pathwise Hopf Formula
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
We develop dual approaches for continuous-time stochastic control problems, enabling the computation of robust dual bounds in high-dimensional state and control spaces. Building on the dual formulation proposed in [L. C. G. Rogers, SIAM Journal on Control and Optimization, 46 (2007), pp. 1116--1132], we first formulate the inner optimization problem as a stochastic partial differential equation (SPDE); the expectation of its solution yields the dual bound. Curse-of-dimensionality-free methods are proposed based on the Pontryagin maximum principle and the generalized Hopf formula. In the process, we prove the generalized Hopf formula, first introduced as a conjecture in [Y. T. Chow, J. Darbon, S. Osher, and W. Yin, Journal of Computational Physics 387 (2019), pp. 376--409], under mild conditions. Numerical experiments demonstrate that our dual approaches effectively complement primal methods, including the deep BSDE method for solving high-dimensional PDEs and the deep actor-critic method in reinforcement learning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 36 references that are in the map. First posted on arXiv 9 Apr 2026.
Builds on
14 of 36 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 191
- 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
- 2020Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path spacecited by 82
- 2016Deep Learning Approximation for Stochastic Control Problemscited by 55
- 2023Recent Developments in Machine Learning Methods for Stochastic Control and Gamescited by 24
- 2022A Neural Network Approach for Stochastic Optimal Controlcited by 13
- 2017Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton-Jacobi equationscited by 9
- 2017Algorithm for Overcoming the Curse of Dimensionality for State-dependent Hamilton-Jacobi equationscited by 7
- 2024A deep primal-dual BSDE method for optimal stopping problemscited by 4
- 2013Stochastic control with rough pathscited by 2
- 2021UVIP: Model-Free Approach to Evaluate Reinforcement Learning Algorithmscited by 1
- 2025Adversarial Reinforcement Learning: A Duality-Based Approach To Solving Optimal Control Problemscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025An Optimization-Based Framework for Solving Forward-Backward Stochastic Differential Equations: Convergence Analysis and Error Bounds
- 2026Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
- 2026The Bias of Nonlinear Two-Time-scale Stochastic Approximation under Constant Step-Sizes
- 2024Markovian Foundations for Quasi-Stochastic Approximation in Two Timescales: Extended Version
- 2025Equivalence of stochastic and deterministic policy gradients
- 2025Convergence of Proximal Policy Gradient Method for Problems with Control Dependent Diffusion Coefficients
- 2026Learning-Based Stochastic Optimal Control with Infinite-Horizon Probabilistic Constraints
- 2026Non-Asymptotic Convergence of Stochastic Iterative Algorithms: A Lyapunov Framework