# Neural-net-induced Gaussian process regression for function approximation and PDE solution

arXiv: https://arxiv.org/abs/1806.11187 · HTML: https://mlsanity.com/paper/1806.11187/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2018.

## Built on by (most cited first, 22 of 22)

- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Learning elliptic partial differential equations with randomized linear algebra](https://mlsanity.com/paper/2102.00491/) (2021)
- [Monte Carlo PINNs: deep learning approach for forward and inverse problems involving high dimensional fractional partial differential equations](https://mlsanity.com/paper/2203.08501/) (2022)
- [Gaussian Process Priors for Systems of Linear Partial Differential Equations with Constant Coefficients](https://mlsanity.com/paper/2212.14319/) (2022)
- [MultiAuto-DeepONet: A Multi-resolution Autoencoder DeepONet for Nonlinear Dimension Reduction, Uncertainty Quantification and Operator Learning of Forward and Inverse Stochastic Problems](https://mlsanity.com/paper/2204.03193/) (2022)
- [Spiking Neural Operators for Scientific Machine Learning](https://mlsanity.com/paper/2205.10130/) (2022)
- [Random Grid Neural Processes for Parametric Partial Differential Equations](https://mlsanity.com/paper/2301.11040/) (2023)
- [On out-of-distribution detection with Bayesian neural networks](https://mlsanity.com/paper/2110.06020/) (2021)
- [Adaptive Uncertainty-Guided Model Selection for Data-Driven PDE Discovery](https://mlsanity.com/paper/2308.10283/) (2023)
- [Neural Network Gaussian Processes by Increasing Depth](https://mlsanity.com/paper/2108.12862/) (2021)
- [A physics-informed neural network framework for modeling obstacle-related equations](https://mlsanity.com/paper/2304.03552/) (2023)
- [On the Correspondence between Gaussian Processes and Geometric Harmonics](https://mlsanity.com/paper/2110.02296/) (2021)
- [PAGP: A physics-assisted Gaussian process framework with active learning for forward and inverse problems of partial differential equations](https://mlsanity.com/paper/2204.02583/) (2022)
- [Solving Maxwell's Equation in 2D with Neural Networks with Local Converging Inputs](https://mlsanity.com/paper/2302.02860/) (2023)
- [Active Learning with Bayesian Multi-Fidelity Laplace Neural Operators for Oscillatory Parametric PDEs](https://mlsanity.com/paper/2502.00550/) (2025)
- [Neural Partial Differential Equations with Functional Convolution](https://mlsanity.com/paper/2303.07194/) (2023)
- [A Unified Kernel for Neural Network Learning](https://mlsanity.com/paper/2403.17467/) (2024)
- [Fredholm Neural Networks for inverse problems in elliptic PDEs](https://mlsanity.com/paper/2507.06038/) (2025)
- [The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach](https://mlsanity.com/paper/2302.08309/) (2023)
- [Reconstructing Blood Flow in Data-Poor Regimes: A Vasculature Network Kernel for Gaussian Process Regression](https://mlsanity.com/paper/2403.09758/) (2024)
- [Uncertainty Quantification From Scaling Laws in Deep Neural Networks](https://mlsanity.com/paper/2503.05938/) (2025)
- [A general physics-constrained method for the modelling of equation's closure terms with sparse data](https://mlsanity.com/paper/2505.03783/) (2025)

## Abstract (arXiv)

Neural-net-induced Gaussian process (NNGP) regression inherits both the high expressivity of deep neural networks (deep NNs) as well as the uncertainty quantification property of Gaussian processes (GPs). We generalize the current NNGP to first include a larger number of hyperparameters and subsequently train the model by maximum likelihood estimation. Unlike previous works on NNGP that targeted classification, here we apply the generalized NNGP to function approximation and to solving partial differential equations (PDEs). Specifically, we develop an analytical iteration formula to compute the covariance function of GP induced by deep NN with an error-function nonlinearity. We compare the performance of the generalized NNGP for function approximations and PDE solutions with those of GPs and fully-connected NNs. We observe that for smooth functions the generalized NNGP can yield the same order of accuracy with GP, while both NNGP and GP outperform deep NN. For non-smooth functions, the generalized NNGP is superior to GP and comparable or superior to deep NN.