# Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes

arXiv: https://arxiv.org/abs/2304.01294 · HTML: https://mlsanity.com/paper/2304.01294/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 89 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 30 of 35)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations](https://mlsanity.com/paper/1703.10230/) (2017)
- [Can Physics-Informed Neural Networks beat the Finite Element Method?](https://mlsanity.com/paper/2302.04107/) (2023)
- [Bayesian Optimization with Gradients](https://mlsanity.com/paper/1703.04389/) (2017)
- [Bayesian Probabilistic Numerical Methods](https://mlsanity.com/paper/1702.03673/) (2017)
- [Kernel Flows: from learning kernels from data into the abyss](https://mlsanity.com/paper/1808.04475/) (2018)
- [Sparse Cholesky factorization by Kullback-Leibler minimization](https://mlsanity.com/paper/2004.14455/) (2020)
- [Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluations](https://mlsanity.com/paper/2207.06503/) (2022)
- [Scaling Gaussian Process Regression with Derivatives](https://mlsanity.com/paper/1810.12283/) (2018)
- [Permutation and Grouping Methods for Sharpening Gaussian Process Approximations](https://mlsanity.com/paper/1609.05372/) (2016)
- [Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity](https://mlsanity.com/paper/1706.02205/) (2017)
- [Vecchia approximations of Gaussian-process predictions](https://mlsanity.com/paper/1805.03309/) (2018)
- [A multi-resolution approximation for massive spatial datasets](https://mlsanity.com/paper/1507.04789/) (2015)
- [Competitive Physics Informed Networks](https://mlsanity.com/paper/2204.11144/) (2022)
- [Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation](https://mlsanity.com/paper/2005.11375/) (2020)
- [High-Dimensional Gaussian Process Inference with Derivatives](https://mlsanity.com/paper/2102.07542/) (2021)
- [Multigrid with rough coefficients and Multiresolution operator decomposition from Hierarchical Information Games](https://mlsanity.com/paper/1503.03467/) (2015)
- [Scaling Gaussian Processes with Derivative Information Using Variational Inference](https://mlsanity.com/paper/2107.04061/) (2021)

## Built on by (most cited first, 30 of 31)

- [Error Analysis of Kernel/GP Methods for Nonlinear and Parametric PDEs](https://mlsanity.com/paper/2305.04962/) (2023)
- [Operator Learning with Gaussian Processes](https://mlsanity.com/paper/2409.04538/) (2024)
- [Solving High Frequency and Multi-Scale PDEs with Gaussian Processes](https://mlsanity.com/paper/2311.04465/) (2023)
- [Linearization Turns Neural Operators into Function-Valued Gaussian Processes](https://mlsanity.com/paper/2406.05072/) (2024)
- [Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification in scientific machine learning](https://mlsanity.com/paper/2404.08809/) (2024)
- [Codiscovering graphical structure and functional relationships within data: A Gaussian Process framework for connecting the dots](https://mlsanity.com/paper/2311.17007/) (2023)
- [Toward Efficient Kernel-Based Solvers for Nonlinear PDEs](https://mlsanity.com/paper/2410.11165/) (2024)
- [Data-Efficient Kernel Methods for Learning Differential Equations and Their Solution Operators: Algorithms and Error Analysis](https://mlsanity.com/paper/2503.01036/) (2025)
- [Operator Learning at Machine Precision](https://mlsanity.com/paper/2511.19980/) (2025)
- [Neural Operator induced Gaussian Process framework for probabilistic solution of parametric partial differential equations](https://mlsanity.com/paper/2404.15618/) (2024)
- [Gaussian Measures Conditioned on Nonlinear Observations: Consistency, MAP Estimators, and Simulation](https://mlsanity.com/paper/2405.13149/) (2024)
- [Towards Gaussian Process for operator learning: an uncertainty aware resolution independent operator learning algorithm for computational mechanics](https://mlsanity.com/paper/2409.10972/) (2024)
- [Solving Nonlinear PDEs with Sparse Radial Basis Function Networks](https://mlsanity.com/paper/2505.07765/) (2025)
- [Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project](https://mlsanity.com/paper/2505.13723/) (2025)
- [Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processes](https://mlsanity.com/paper/2510.05568/) (2025)
- [Decoding Mean Field Games from Population and Environment Observations By Gaussian Processes](https://mlsanity.com/paper/2312.06625/) (2023)
- [Scaling up Probabilistic PDE Simulators with Structured Volumetric Information](https://mlsanity.com/paper/2406.05020/) (2024)
- [Learning Surrogate Potential Mean Field Games via Gaussian Processes: A Data-Driven Approach to Ill-Posed Inverse Problems](https://mlsanity.com/paper/2502.11506/) (2025)
- [Physics-informed, boundary-constrained Gaussian process regression for the reconstruction of fluid flow fields](https://mlsanity.com/paper/2507.17582/) (2025)
- [LVM-GP: Uncertainty-Aware PDE Solver via coupling latent variable model and Gaussian process](https://mlsanity.com/paper/2507.22493/) (2025)
- [Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs](https://mlsanity.com/paper/2510.13772/) (2025)
- [Sparse RBF Networks for PDEs and nonlocal equations: function space theory, operator calculus, and training algorithms](https://mlsanity.com/paper/2601.17562/) (2026)
- [KROM: Kernelized Reduced Order Modeling](https://mlsanity.com/paper/2603.00360/) (2026)
- [Linear Stability Analysis of Physics-Informed Random Projection Neural Networks for ODEs](https://mlsanity.com/paper/2408.15393/) (2024)
- [An interpretation of the Brownian bridge as a physics-informed prior for the Poisson equation](https://mlsanity.com/paper/2503.00213/) (2025)
- [Gaussian process policy iteration with additive Schwarz acceleration for forward and inverse HJB and mean field game problems](https://mlsanity.com/paper/2505.00909/) (2025)
- [A Streaming Sparse Cholesky Method for Derivative-Informed Gaussian Process Surrogates Within Digital Twin Applications](https://mlsanity.com/paper/2511.00366/) (2025)
- [Scalable Bayesian Inference for Nonlinear Conservation Laws](https://mlsanity.com/paper/2605.31127/) (2026)
- [Extrapolating from Regularised Solutions for Solving Ill-Conditioned Linear Systems in Machine Learning](https://mlsanity.com/paper/2606.30328/) (2026)
- [Physics Informed Random Feature Neural Networks for Solving PDEs](https://mlsanity.com/paper/2609.16406/) (2026)

## Abstract (arXiv)

In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its textitpartial derivatives, a result of the PDE constraint and for which fast algorithms are scarce. The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute ε-approximate inverse Cholesky factors of the kernel matrices with complexity O(Nlogd(N/ε)) in space and O(Nlog2d(N/ε)) in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm's near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Amp`ere equations.