# Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks

arXiv: https://arxiv.org/abs/2102.11379 · HTML: https://mlsanity.com/paper/2102.11379/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2021.

## Built on by (most cited first, 29 of 29)

- [An optimal control perspective on diffusion-based generative modeling](https://mlsanity.com/paper/2211.01364/) (2022)
- [Stochastic Optimal Control Matching](https://mlsanity.com/paper/2312.02027/) (2023)
- [Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference](https://mlsanity.com/paper/2508.12511/) (2025)
- [Interpolating between BSDEs and PINNs: deep learning for elliptic and parabolic boundary value problems](https://mlsanity.com/paper/2112.03749/) (2021)
- [Robust SDE-Based Variational Formulations for Solving Linear PDEs via Deep Learning](https://mlsanity.com/paper/2206.10588/) (2022)
- [A Taxonomy of Loss Functions for Stochastic Optimal Control](https://mlsanity.com/paper/2410.00345/) (2024)
- [A Policy Gradient Framework for Stochastic Optimal Control Problems with Global Convergence Guarantee](https://mlsanity.com/paper/2302.05816/) (2023)
- [SympOCnet: Solving optimal control problems with applications to high-dimensional multi-agent path planning problems](https://mlsanity.com/paper/2201.05475/) (2022)
- [Neural Network Approaches for Parameterized Optimal Control](https://mlsanity.com/paper/2402.10033/) (2024)
- [On Bellman equations for continuous-time policy evaluation I: discretization and approximation](https://mlsanity.com/paper/2407.05966/) (2024)
- [Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problems](https://mlsanity.com/paper/2409.16471/) (2024)
- [From continuous-time formulations to discretization schemes: tensor trains and robust regression for BSDEs and parabolic PDEs](https://mlsanity.com/paper/2307.15496/) (2023)
- [A brief review of the Deep BSDE method for solving high-dimensional partial differential equations](https://mlsanity.com/paper/2505.17032/) (2025)
- [One model to solve them all: 2BSDE families via neural operators](https://mlsanity.com/paper/2511.01125/) (2025)
- [Generative Modelling with Tensor Train approximations of Hamilton--Jacobi--Bellman equations](https://mlsanity.com/paper/2402.15285/) (2024)
- [Neural Actor-Critic Methods for Hamilton-Jacobi-Bellman PDEs: Asymptotic Analysis and Numerical Studies](https://mlsanity.com/paper/2507.06428/) (2025)
- [Drift Control of High-Dimensional RBM: A Computational Method Based on Neural Networks](https://mlsanity.com/paper/2309.11651/) (2023)
- [Convergence of the Deep Galerkin Method for Finite State Mean Field Control Problems](https://mlsanity.com/paper/2405.13346/) (2024)
- [Reinforcement Learning with Random Time Horizons](https://mlsanity.com/paper/2506.00962/) (2025)
- [Deep Hilbert--Galerkin Methods for Infinite-Dimensional PDEs and Optimal Control](https://mlsanity.com/paper/2603.19463/) (2026)
- [Deep Neural networks for solving high-dimensional parabolic partial differential equations](https://mlsanity.com/paper/2601.13256/) (2026)
- [Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process](https://mlsanity.com/paper/2606.28669/) (2026)
- [Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models](https://mlsanity.com/paper/2606.28671/) (2026)
- [Control, Transport and Sampling: Towards Better Loss Design](https://mlsanity.com/paper/2405.13731/) (2024)
- [DeepPAAC: A New Deep Galerkin Method for Principal-Agent Problems](https://mlsanity.com/paper/2511.04309/) (2025)
- [Continuous-time reinforcement learning: ellipticity enables model-free value function approximation](https://mlsanity.com/paper/2602.06930/) (2026)
- [An Actor-Critic Framework for Continuous-Time Jump-Diffusion Controls with Normalizing Flows](https://mlsanity.com/paper/2604.05398/) (2026)
- [Viscosity-Informed Generative Actor-Critic for High-Dimensional Stochastic Optimal Control](https://mlsanity.com/paper/2605.15668/) (2026)
- [Self-supervised In-context Operator Learning for Stochastic Mean-Field Control](https://mlsanity.com/paper/2608.18282/) (2026)

## Abstract (arXiv)

We propose a novel numerical method for high dimensional Hamilton--Jacobi--Bellman (HJB) type elliptic partial differential equations (PDEs). The HJB PDEs, reformulated as optimal control problems, are tackled by the actor-critic framework inspired by reinforcement learning, based on neural network parametrization of the value and control functions. Within the actor-critic framework, we employ a policy gradient approach to improve the control, while for the value function, we derive a variance reduced least-squares temporal difference method using stochastic calculus. To numerically discretize the stochastic control problem, we employ an adaptive step size scheme to improve the accuracy near the domain boundary. Numerical examples up to 20 spatial dimensions including the linear quadratic regulators, the stochastic Van der Pol oscillators, the diffusive Eikonal equations, and fully nonlinear elliptic PDEs derived from a regulator problem are presented to validate the effectiveness of our proposed method.