# Neural Q-learning for solving PDEs

arXiv: https://arxiv.org/abs/2203.17128 · HTML: https://mlsanity.com/paper/2203.17128/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 57 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent](https://mlsanity.com/paper/1902.06720/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [On Lazy Training in Differentiable Programming](https://mlsanity.com/paper/1812.07956/) (2018)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit](https://mlsanity.com/paper/1902.06015/) (2019)
- [When Do Neural Networks Outperform Kernel Methods?](https://mlsanity.com/paper/2006.13409/) (2020)
- [Physics-Informed Neural Networks for Power Systems](https://mlsanity.com/paper/1911.03737/) (2019)
- [Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning](https://mlsanity.com/paper/2008.13333/) (2020)
- [An overview on deep learning-based approximation methods for partial differential equations](https://mlsanity.com/paper/2012.12348/) (2020)
- [On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs](https://mlsanity.com/paper/2004.01806/) (2020)
- [Neural networks-based algorithms for stochastic control and PDEs in finance](https://mlsanity.com/paper/2101.08068/) (2021)
- [Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditions](https://mlsanity.com/paper/2007.05384/) (2020)
- [MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs](https://mlsanity.com/paper/2107.03673/) (2021)
- [Neural networks-based backward scheme for fully nonlinear PDEs](https://mlsanity.com/paper/1908.00412/) (2019)
- [PDE-constrained Models with Neural Network Terms: Optimization and Global Convergence](https://mlsanity.com/paper/2105.08633/) (2021)
- [DeepSets and their derivative networks for solving symmetric PDEs](https://mlsanity.com/paper/2103.00838/) (2021)

## Built on by (most cited first, 8 of 8)

- [A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression](https://mlsanity.com/paper/2410.17796/) (2024)
- [The Hard-Constraint PINNs for Interface Optimal Control Problems](https://mlsanity.com/paper/2308.06709/) (2023)
- [Neural Actor-Critic Methods for Hamilton-Jacobi-Bellman PDEs: Asymptotic Analysis and Numerical Studies](https://mlsanity.com/paper/2507.06428/) (2025)
- [Deep Hilbert--Galerkin Methods for Infinite-Dimensional PDEs and Optimal Control](https://mlsanity.com/paper/2603.19463/) (2026)
- [Random Gradient-Free Optimization in Infinite Dimensional Spaces](https://mlsanity.com/paper/2512.20566/) (2025)
- [Enforcing Dirichlet Boundary Conditions in Operator Learning](https://mlsanity.com/paper/2608.27256/) (2026)
- [The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach](https://mlsanity.com/paper/2302.08309/) (2023)
- [Global Convergence of DGM and PINN Algorithms for Solving Nonlinear PDEs](https://mlsanity.com/paper/2607.24726/) (2026)

## Abstract (arXiv)

Solving high-dimensional partial differential equations (PDEs) is a major challenge in scientific computing. We develop a new numerical method for solving elliptic-type PDEs by adapting the Q-learning algorithm in reinforcement learning. Our "Q-PDE" algorithm is mesh-free and therefore has the potential to overcome the curse of dimensionality. Using a neural tangent kernel (NTK) approach, we prove that the neural network approximator for the PDE solution, trained with the Q-PDE algorithm, converges to the trajectory of an infinite-dimensional ordinary differential equation (ODE) as the number of hidden units rightarrow ∞. For monotone PDE (i.e. those given by monotone operators, which may be nonlinear), despite the lack of a spectral gap in the NTK, we then prove that the limit neural network, which satisfies the infinite-dimensional ODE, converges in L2 to the PDE solution as the training time rightarrow ∞. More generally, we can prove that any fixed point of the wide-network limit for the Q-PDE algorithm is a solution of the PDE (not necessarily under the monotone condition). The numerical performance of the Q-PDE algorithm is studied for several elliptic PDEs.