# Improving Output Uncertainty Estimation and Generalization in Deep Learning via Neural Network Gaussian Processes

arXiv: https://arxiv.org/abs/1707.05922 · HTML: https://mlsanity.com/paper/1707.05922/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2017.

## Built on by (most cited first, 14 of 14)

- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice](https://mlsanity.com/paper/2410.15326/) (2024)
- [Simultaneously Reconciled Quantile Forecasting of Hierarchically Related Time Series](https://mlsanity.com/paper/2102.12612/) (2021)
- [Operator Learning with Gaussian Processes](https://mlsanity.com/paper/2409.04538/) (2024)
- [Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Tasks](https://mlsanity.com/paper/2505.20047/) (2025)
- [A Gaussian Process Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2401.03492/) (2024)
- [A Novel Regression Loss for Non-Parametric Uncertainty Optimization](https://mlsanity.com/paper/2101.02726/) (2021)
- [Calibrated simplex-mapping classification](https://mlsanity.com/paper/2103.02926/) (2021)
- [Dynamic Combination of Heterogeneous Models for Hierarchical Time Series](https://mlsanity.com/paper/2112.11669/) (2021)
- [Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling](https://mlsanity.com/paper/2410.04988/) (2024)
- [Photoelectric Factor Prediction Using Automated Learning and Uncertainty Quantification](https://mlsanity.com/paper/2206.08950/) (2022)
- [SEEK: Self-adaptive Explainable Kernel For Nonstationary Gaussian Processes](https://mlsanity.com/paper/2503.14785/) (2025)
- [Semantic-Aware Gaussian Process Calibration with Structured Layerwise Kernels for Deep Neural Networks](https://mlsanity.com/paper/2507.15987/) (2025)

## Abstract (arXiv)

We propose a simple method that combines neural networks and Gaussian processes. The proposed method can estimate the uncertainty of outputs and flexibly adjust target functions where training data exist, which are advantages of Gaussian processes. The proposed method can also achieve high generalization performance for unseen input configurations, which is an advantage of neural networks. With the proposed method, neural networks are used for the mean functions of Gaussian processes. We present a scalable stochastic inference procedure, where sparse Gaussian processes are inferred by stochastic variational inference, and the parameters of neural networks and kernels are estimated by stochastic gradient descent methods, simultaneously. We use two real-world spatio-temporal data sets to demonstrate experimentally that the proposed method achieves better uncertainty estimation and generalization performance than neural networks and Gaussian processes.