# Solving nonconvex Hamilton--Jacobi--Isaacs equations with PINN-based policy iteration

arXiv: https://arxiv.org/abs/2507.15455 · HTML: https://mlsanity.com/paper/2507.15455/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 28 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Builds on (most cited first, 7 of 7)

- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [DeepReach: A Deep Learning Approach to High-Dimensional Reachability](https://mlsanity.com/paper/2011.02082/) (2020)
- [Policy Iteration for Exploratory Hamilton--Jacobi--Bellman Equations](https://mlsanity.com/paper/2406.00612/) (2024)
- [Policy iteration for the deterministic control problems -- a viscosity approach](https://mlsanity.com/paper/2301.00419/) (2023)
- [Hamilton-Jacobi Based Policy-Iteration via Deep Operator Learning](https://mlsanity.com/paper/2406.10920/) (2024)
- [Policy iteration for nonconvex viscous Hamilton--Jacobi equations](https://mlsanity.com/paper/2503.02159/) (2025)

## Built on by (most cited first, 1 of 1)

- [Monotone Neural Policy Iteration for High-Dimensional First-Order Hamilton--Jacobi--Bellman Equations](https://mlsanity.com/paper/2605.07116/) (2026)

## Abstract (arXiv)

We propose a mesh-free policy iteration framework that combines classical dynamic programming with physics-informed neural networks (PINNs) to solve high-dimensional, nonconvex Hamilton--Jacobi--Isaacs (HJI) equations arising in stochastic differential games and robust control. The method alternates between solving linear second-order PDEs under fixed feedback policies and updating the controls via pointwise minimax optimization using automatic differentiation. Under standard Lipschitz and uniform ellipticity assumptions, we prove that the value function iterates converge locally uniformly to the unique viscosity solution of the HJI equation. The analysis establishes equi-Lipschitz regularity of the iterates, enabling provable stability and convergence without requiring convexity of the Hamiltonian. Numerical experiments demonstrate the accuracy and scalability of the method. In a two-dimensional stochastic path-planning game with a moving obstacle, our method matches finite-difference benchmarks with relative L2-errors below %10-2%. In five- and ten-dimensional publisher-subscriber differential games with anisotropic noise, the proposed approach consistently outperforms direct PINN solvers, yielding smoother value functions and lower residuals. Our results suggest that integrating PINNs with policy iteration is a practical and theoretically grounded method for solving high-dimensional, nonconvex HJI equations, with potential applications in robotics, finance, and multi-agent reinforcement learning.