# Interpretable deep Gaussian processes with moments

arXiv: https://arxiv.org/abs/1905.10963 · HTML: https://mlsanity.com/paper/1905.10963/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2019.

## Built on by (most cited first, 8 of 8)

- [The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective](https://mlsanity.com/paper/2106.06529/) (2021)
- [Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes](https://mlsanity.com/paper/2309.12658/) (2023)
- [Neural Network Gaussian Processes by Increasing Depth](https://mlsanity.com/paper/2108.12862/) (2021)
- [Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN](https://mlsanity.com/paper/2204.10970/) (2022)
- [Conditional Deep Gaussian Processes: empirical Bayes hyperdata learning](https://mlsanity.com/paper/2110.00568/) (2021)
- [On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel](https://mlsanity.com/paper/2203.07411/) (2022)
- [Bayesian inference with finitely wide neural networks](https://mlsanity.com/paper/2303.02859/) (2023)
- [How Deep Are Deep GPs, Really? A Sharp Threshold and a Non-Gaussian Limit for Compositional GPs](https://mlsanity.com/paper/2606.08218/) (2026)

## Abstract (arXiv)

Deep Gaussian Processes (DGPs) combine the expressiveness of Deep Neural Networks (DNNs) with quantified uncertainty of Gaussian Processes (GPs). Expressive power and intractable inference both result from the non-Gaussian distribution over composition functions. We propose interpretable DGP based on approximating DGP as a GP by calculating the exact moments, which additionally identify the heavy-tailed nature of some DGP distributions. Consequently, our approach admits interpretation as both NNs with specified activation functions and as a variational approximation to DGP. We identify the expressivity parameter of DGP and find non-local and non-stationary correlation from DGP composition. We provide general recipes for deriving the effective kernels for DGP of two, three, or infinitely many layers, composed of homogeneous or heterogeneous kernels. Results illustrate the expressiveness of our effective kernels through samples from the prior and inference on simulated and real data and demonstrate advantages of interpretability by analysis of analytic forms, and draw relations and equivalences across kernels.