# A Neural Network Approach for Stochastic Optimal Control

arXiv: https://arxiv.org/abs/2209.13104 · HTML: https://mlsanity.com/paper/2209.13104/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2022.

## Built on by (most cited first, 13 of 13)

- [Neural Network Approaches for Parameterized Optimal Control](https://mlsanity.com/paper/2402.10033/) (2024)
- [On Bellman equations for continuous-time policy evaluation I: discretization and approximation](https://mlsanity.com/paper/2407.05966/) (2024)
- [Generative Neural Operators of Log-Complexity Can Simultaneously Solve Infinitely Many Convex Programs](https://mlsanity.com/paper/2508.14995/) (2025)
- [Deep Learning for Continuous-Time Stochastic Control with Jumps](https://mlsanity.com/paper/2505.15602/) (2025)
- [End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagation](https://mlsanity.com/paper/2510.00359/) (2025)
- [Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems](https://mlsanity.com/paper/2509.18404/) (2025)
- [PINF: Continuous Normalizing Flows for Physics-Constrained Deep Learning](https://mlsanity.com/paper/2309.15139/) (2023)
- [Solving a class of stochastic optimal control problems by physics-informed neural networks](https://mlsanity.com/paper/2402.15592/) (2024)
- [Barron Space Representations for Elliptic PDEs with Homogeneous Boundary Conditions](https://mlsanity.com/paper/2508.07559/) (2025)
- [Deep Neural networks for solving high-dimensional parabolic partial differential equations](https://mlsanity.com/paper/2601.13256/) (2026)
- [On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonians](https://mlsanity.com/paper/2602.00921/) (2026)
- [Learning Control Policies of Hodgkin-Huxley Neuronal Dynamics](https://mlsanity.com/paper/2311.07563/) (2023)
- [Dual Approaches to Stochastic Control via SPDEs and the Pathwise Hopf Formula](https://mlsanity.com/paper/2604.08155/) (2026)

## Abstract (arXiv)

We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semi-linear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the non-viscous Hamilton-Jacobi-Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution and, via a modification, we show the wider applicability of our scheme.