mlsanityData to 7 Oct 2026

Sparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes

arXiv 2304.01294 ↗First posted 3 Apr 2023math.NA cs.NA math.OCcited by 31

Not on the map. Why?

Abstract (arXiv)

In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel methods have garnered considerable interest due to their flexibility, robust theoretical guarantees, and close ties to traditional methods. They can transform the solving of general nonlinear PDEs into solving quadratic optimization problems with nonlinear, PDE-induced constraints. However, the complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel, and its textitpartial derivatives, a result of the PDE constraint and for which fast algorithms are scarce. The primary goal of this paper is to provide a near-linear complexity algorithm for working with such kernel matrices. We present a sparse Cholesky factorization algorithm for these matrices based on the near-sparsity of the Cholesky factor under a novel ordering of pointwise and derivative measurements. The near-sparsity is rigorously justified by directly connecting the factor to GP regression and exponential decay of basis functions in numerical homogenization. We then employ the Vecchia approximation of GPs, which is optimal in the Kullback-Leibler divergence, to compute the approximate factor. This enables us to compute ε-approximate inverse Cholesky factors of the kernel matrices with complexity O(Nlogd(N/ε)) in space and O(Nlog2d(N/ε)) in time. We integrate sparse Cholesky factorizations into optimization algorithms to obtain fast solvers of the nonlinear PDE. We numerically illustrate our algorithm's near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Amp`ere equations.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 35 of its 89 references that are in the map. First posted on arXiv 3 Apr 2023.

Builds on

35 of 89 references.

References · 35

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  4. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  5. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  6. 2017Deep Neural Networks as Gaussian Processescited by 617
  7. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  8. 2015Deep Kernel Learningcited by 440
  9. 2020Model Reduction and Neural Networks for Parametric PDEscited by 257
  10. 2018When Gaussian Process Meets Big Data: A Review of Scalable GPscited by 205
  11. 2015Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)cited by 175
  12. 2021Solving and Learning Nonlinear PDEs with Gaussian Processescited by 128
23 more
  1. 2020The Random Feature Model for Input-Output Maps between Banach Spacescited by 90
  2. 2017Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equationscited by 72
  3. 2023Can Physics-Informed Neural Networks beat the Finite Element Method?cited by 68
  4. 2017Bayesian Optimization with Gradientscited by 59
  5. 2017Bayesian Probabilistic Numerical Methodscited by 50
  6. 2018Kernel Flows: from learning kernels from data into the abysscited by 49
  7. 2020Sparse Cholesky factorization by Kullback-Leibler minimizationcited by 40
  8. 2022Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluationscited by 31
  9. 2018Scaling Gaussian Process Regression with Derivativescited by 27
  10. 2016Permutation and Grouping Methods for Sharpening Gaussian Process Approximationscited by 24
  11. 2017Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexitycited by 24
  12. 2018Vecchia approximations of Gaussian-process predictionscited by 21
  13. 2015A multi-resolution approximation for massive spatial datasetscited by 20
  14. 2022Competitive Physics Informed Networkscited by 19
  15. 2020Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimationcited by 18
  16. 2021High-Dimensional Gaussian Process Inference with Derivativescited by 16
  17. 2015Multigrid with rough coefficients and Multiresolution operator decomposition from Hierarchical Information Gamescited by 15
  18. 2021Scaling Gaussian Processes with Derivative Information Using Variational Inferencecited by 15
  19. 2022One-Shot Learning of Stochastic Differential Equations with Data Adapted Kernelscited by 15
  20. 2018Scalable Gaussian Process Computations Using Hierarchical Matricescited by 8
  21. 2022A Kernel Approach for PDE Discovery and Operator Learningcited by 7
  22. 2016Fast spatial Gaussian process maximum likelihood estimation via skeletonization factorizationscited by 6
  23. 2020The SPDE Approach to Mat'ern Fields: Graph Representationscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 9; 2025: 12; 2026: 6.

31 papers.

Sort:

  1. 2023Error Analysis of Kernel/GP Methods for Nonlinear and Parametric PDEscited by 21
  2. 2024Operator Learning with Gaussian Processescited by 18
  3. 2023Solving High Frequency and Multi-Scale PDEs with Gaussian Processescited by 14
  4. 2024Linearization Turns Neural Operators into Function-Valued Gaussian Processescited by 11
  5. 2024Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification in scientific machine learningcited by 9
  6. 2023Codiscovering graphical structure and functional relationships within data: A Gaussian Process framework for connecting the dotscited by 8
  7. 2024Toward Efficient Kernel-Based Solvers for Nonlinear PDEscited by 7
  8. 2025Data-Efficient Kernel Methods for Learning Differential Equations and Their Solution Operators: Algorithms and Error Analysiscited by 6
  9. 2025Operator Learning at Machine Precisioncited by 6
  10. 2024Neural Operator induced Gaussian Process framework for probabilistic solution of parametric partial differential equationscited by 4
  11. 2024Gaussian Measures Conditioned on Nonlinear Observations: Consistency, MAP Estimators, and Simulationcited by 4
  12. 2024Towards Gaussian Process for operator learning: an uncertainty aware resolution independent operator learning algorithm for computational mechanicscited by 3
19 more
  1. 2025Solving Nonlinear PDEs with Sparse Radial Basis Function Networkscited by 3
  2. 2025Turbocharging Gaussian Process Inference with Approximate Sketch-and-Projectcited by 3
  3. 2025Bilevel optimization for learning hyperparameters: Application to solving PDEs and inverse problems with Gaussian processescited by 3
  4. 2023Decoding Mean Field Games from Population and Environment Observations By Gaussian Processescited by 2
  5. 2024Scaling up Probabilistic PDE Simulators with Structured Volumetric Informationcited by 2
  6. 2025Learning Surrogate Potential Mean Field Games via Gaussian Processes: A Data-Driven Approach to Ill-Posed Inverse Problemscited by 2
  7. 2025Physics-informed, boundary-constrained Gaussian process regression for the reconstruction of fluid flow fieldscited by 1
  8. 2025LVM-GP: Uncertainty-Aware PDE Solver via coupling latent variable model and Gaussian processcited by 1
  9. 2025Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEscited by 1
  10. 2026KROM: Kernelized Reduced Order Modelingcited by 1
  11. 2024Linear Stability Analysis of Physics-Informed Random Projection Neural Networks for ODEscited by 0
  12. 2025An interpretation of the Brownian bridge as a physics-informed prior for the Poisson equationcited by 0
  13. 2025Gaussian process policy iteration with additive Schwarz acceleration for forward and inverse HJB and mean field game problemscited by 0
  14. 2025A Streaming Sparse Cholesky Method for Derivative-Informed Gaussian Process Surrogates Within Digital Twin Applicationscited by 0
  15. 2026Sparse RBF Networks for PDEs and nonlocal equations: function space theory, operator calculus, and training algorithmscited by 0
  16. 2026Scalable Bayesian Inference for Nonlinear Conservation Lawscited by 0
  17. 2026Extrapolating from Regularised Solutions for Solving Ill-Conditioned Linear Systems in Machine Learningcited by 0
  18. 2026Physics Informed Random Feature Neural Networks for Solving PDEscited by 0
  19. 2026Adaptive multi-resolution Gaussian processes: Scalable exact inference with naturally data-sparse covariance matricescited by 0