# Monotone Neural Policy Iteration for High-Dimensional First-Order Hamilton--Jacobi--Bellman Equations

arXiv: https://arxiv.org/abs/2605.07116 · HTML: https://mlsanity.com/paper/2605.07116/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 8 May 2026.

## Builds on (most cited first, 14 of 14)

- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations](https://mlsanity.com/paper/1907.05317/) (2019)
- [Algorithms for Overcoming the Curse of Dimensionality for Certain Hamilton-Jacobi Equations Arising in Control Theory and Elsewhere](https://mlsanity.com/paper/1605.01799/) (2016)
- [Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures](https://mlsanity.com/paper/1910.09045/) (2019)
- [Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification](https://mlsanity.com/paper/2402.10119/) (2024)
- [Policy Iteration for Exploratory Hamilton--Jacobi--Bellman Equations](https://mlsanity.com/paper/2406.00612/) (2024)
- [Algorithm for Overcoming the Curse of Dimensionality for State-dependent Hamilton-Jacobi equations](https://mlsanity.com/paper/1704.02524/) (2017)
- [Policy iteration for the deterministic control problems -- a viscosity approach](https://mlsanity.com/paper/2301.00419/) (2023)
- [Hamilton-Jacobi Based Policy-Iteration via Deep Operator Learning](https://mlsanity.com/paper/2406.10920/) (2024)
- [Finite-difference least square methods for solving Hamilton-Jacobi equations using neural networks](https://mlsanity.com/paper/2406.10758/) (2024)
- [Policy iteration for nonconvex viscous Hamilton--Jacobi equations](https://mlsanity.com/paper/2503.02159/) (2025)
- [Solving nonconvex Hamilton--Jacobi--Isaacs equations with PINN-based policy iteration](https://mlsanity.com/paper/2507.15455/) (2025)
- [Physics-Informed Policy Iteration for High-Dimensional Hamilton--Jacobi--Bellman Equations: Interior Error Bounds without Boundary Data](https://mlsanity.com/paper/2508.01718/) (2025)
- [Solving Hamilton-Jacobi equations by residual minimization of monotone finite-difference discretizations](https://mlsanity.com/paper/2601.21764/) (2026)

## Abstract (arXiv)

We analyze a neural semi-discrete method for high-dimensional first-order Hamilton-Jacobi-Bellman (HJB) equations with known or learned dynamics. Centered differences and an artificial viscosity Nh=O(h) define a monotone operator evaluated through 2d+1 shifted network queries; policy iteration solves the resulting Bellman equation without a tensor grid. At fixed h, the sharp componentwise condition maxi|fi|≤2N turns every frozen-policy operator into a nearest-neighbor Markov-chain generator with a policy-independent total jump rate. Uniformization gives whole-space well-posedness for measurable feedbacks, an explicit Poisson-tail bound on the numerical domain of dependence, and boundary-free localization. The representation also yields a posteriori policy-evaluation bounds that account for residual and learned-model errors. A greedy-gap analysis controls inexact policy iteration at fixed h; a separate consistency estimate connects the semi-discrete equation to the continuous HJB equation. Experiments reproduce the extremal tail, show rates consistent with O(√ h) and nearly h-independent exact-policy-iteration decay, and assess empirical estimator effectivity. A nonsmooth example shows that the continuous residual can miss a non-viscosity solution, whereas the shifted residual detects the defect. Further tests provide a structured interval-verified certificate calibration, an early-budget benefit of policy freezing for bang-bang control, and learned-dynamics diagnostics. A structured nonlinear problem with active compact-control constraints is tested against a manufactured semi-discrete reference through d=1024.