# Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification

arXiv: https://arxiv.org/abs/2402.10119 · HTML: https://mlsanity.com/paper/2402.10119/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2024.

## Built on by (most cited first, 13 of 13)

- [Actor-Critic Physics-informed Neural Lyapunov Control](https://mlsanity.com/paper/2403.08448/) (2024)
- [Formally Verified Physics-Informed Neural Control Lyapunov Functions](https://mlsanity.com/paper/2409.20528/) (2024)
- [Stochastic Reinforcement Learning with Stability Guarantees for Control of Unknown Nonlinear Systems](https://mlsanity.com/paper/2409.08382/) (2024)
- [Continuous-Time Value Iteration for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2509.09135/) (2025)
- [Kernel-Based Optimal Control: An Infinitesimal Generator Approach](https://mlsanity.com/paper/2412.01591/) (2024)
- [Physics-Informed Neural Networks for Optimal Vaccination Plan in SIR Epidemic Models](https://mlsanity.com/paper/2502.19890/) (2025)
- [A Physics-preserved Transfer Learning Method for Differential Equations](https://mlsanity.com/paper/2505.01281/) (2025)
- [Physics-Informed Policy Iteration for High-Dimensional Hamilton--Jacobi--Bellman Equations: Interior Error Bounds without Boundary Data](https://mlsanity.com/paper/2508.01718/) (2025)
- [Rigorous Error Certification for Neural PDE Solvers: From Empirical Residuals to Solution Guarantees](https://mlsanity.com/paper/2603.19165/) (2026)
- [Verifiable Error Bounds for Physics-Informed Neural Network Solutions of Lyapunov and Hamilton-Jacobi-Bellman Equations](https://mlsanity.com/paper/2603.19545/) (2026)
- [Monotone Neural Policy Iteration for High-Dimensional First-Order Hamilton--Jacobi--Bellman Equations](https://mlsanity.com/paper/2605.07116/) (2026)
- [Physics-informed Reinforcement Learning for Stochastic Reach-Avoid Analysis](https://mlsanity.com/paper/2608.17117/) (2026)
- [Learning Lyapunov Operators for Nonlinear Systems](https://mlsanity.com/paper/2609.18894/) (2026)

## Abstract (arXiv)

Solving nonlinear optimal control problems is a challenging task, particularly for high-dimensional problems. We propose algorithms for model-based policy iterations to solve nonlinear optimal control problems with convergence guarantees. The main component of our approach is an iterative procedure that utilizes neural approximations to solve linear partial differential equations (PDEs), ensuring convergence. We present two variants of the algorithms. The first variant formulates the optimization problem as a linear least square problem, drawing inspiration from extreme learning machine (ELM) for solving PDEs. This variant efficiently handles low-dimensional problems with high accuracy. The second variant is based on a physics-informed neural network (PINN) for solving PDEs and has the potential to address high-dimensional problems. We demonstrate that both algorithms outperform traditional approaches, such as Galerkin methods, by a significant margin. We provide a theoretical analysis of both algorithms in terms of convergence of neural approximations towards the true optimal solutions in a general setting. Furthermore, we employ formal verification techniques to demonstrate the verifiable stability of the resulting controllers.