# Random Feature Expansions for Deep Gaussian Processes

arXiv: https://arxiv.org/abs/1610.04386 · HTML: https://mlsanity.com/paper/1610.04386/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2016.

## Built on by (most cited first, 30 of 32)

- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Deep Neural Networks as Point Estimates for Deep Gaussian Processes](https://mlsanity.com/paper/2105.04504/) (2021)
- [The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective](https://mlsanity.com/paper/2106.06529/) (2021)
- [Bayesian Calibration of Imperfect Computer Models using Physics-Informed Priors](https://mlsanity.com/paper/2201.06463/) (2022)
- [Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features](https://mlsanity.com/paper/2106.05960/) (2021)
- [Deep Variational Implicit Processes](https://mlsanity.com/paper/2206.06720/) (2022)
- [A Unifying Perspective on Non-Stationary Kernels for Deeper Gaussian Processes](https://mlsanity.com/paper/2309.10068/) (2023)
- [Sparse Variational Student-t Processes](https://mlsanity.com/paper/2312.05568/) (2023)
- [Free-Form Variational Inference for Gaussian Process State-Space Models](https://mlsanity.com/paper/2302.09921/) (2023)
- [A Sparse Expansion For Deep Gaussian Processes](https://mlsanity.com/paper/2112.05888/) (2021)
- [Sequential Estimation of Gaussian Process-based Deep State-Space Models](https://mlsanity.com/paper/2301.12528/) (2023)
- [On permutation symmetries in Bayesian neural network posteriors: a variational perspective](https://mlsanity.com/paper/2310.10171/) (2023)
- [Cross-Space Adaptive Filter: Integrating Graph Topology and Node Attributes for Alleviating the Over-smoothing Problem](https://mlsanity.com/paper/2401.14876/) (2024)
- [Convolutional Normalizing Flows for Deep Gaussian Processes](https://mlsanity.com/paper/2104.08472/) (2021)
- [Gaussian Process-Gated Hierarchical Mixtures of Experts](https://mlsanity.com/paper/2302.04947/) (2023)
- [Trigonometric Quadrature Fourier Features for Scalable Gaussian Process Regression](https://mlsanity.com/paper/2310.14544/) (2023)
- [Sparse Variational Student-t Processes for Heavy-tailed Modeling](https://mlsanity.com/paper/2408.06699/) (2024)
- [From Deep Additive Kernel Learning to Last-Layer Bayesian Neural Networks via Induced Prior Approximation](https://mlsanity.com/paper/2502.10540/) (2025)
- [Scalable Random Wavelet Features: Efficient Non-Stationary Kernel Approximation with Convergence Guarantees](https://mlsanity.com/paper/2602.00987/) (2026)
- [Explaining Deep Tractable Probabilistic Models: The sum-product network case](https://mlsanity.com/paper/2110.09778/) (2021)
- [On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel](https://mlsanity.com/paper/2203.07411/) (2022)
- [Bayesian inference with finitely wide neural networks](https://mlsanity.com/paper/2303.02859/) (2023)
- [Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learning](https://mlsanity.com/paper/2311.14828/) (2023)
- [Adaptive RKHS Fourier Features for Compositional Gaussian Process Models](https://mlsanity.com/paper/2407.01856/) (2024)
- [Sparse Techniques for Regression in Deep Gaussian Processes](https://mlsanity.com/paper/2505.11355/) (2025)
- [Prostate Tissue Grading with Deep Quantum Measurement Ordinal Regression](https://mlsanity.com/paper/2103.03188/) (2021)
- [Subset-of-Data Variational Inference for Deep Gaussian-Processes Regression](https://mlsanity.com/paper/2107.08265/) (2021)
- [How Good are Low-Rank Approximations in Gaussian Process Regression?](https://mlsanity.com/paper/2112.06410/) (2021)
- [Locally Smoothed Gaussian Process Regression](https://mlsanity.com/paper/2210.09998/) (2022)
- [Stein Random Feature Regression](https://mlsanity.com/paper/2406.00438/) (2024)

## Abstract (arXiv)

The composition of multiple Gaussian Processes as a Deep Gaussian Process (DGP) enables a deep probabilistic nonparametric approach to flexibly tackle complex machine learning problems with sound quantification of uncertainty. Existing inference approaches for DGP models have limited scalability and are notoriously cumbersome to construct. In this work, we introduce a novel formulation of DGPs based on random feature expansions that we train using stochastic variational inference. This yields a practical learning framework which significantly advances the state-of-the-art in inference for DGPs, and enables accurate quantification of uncertainty. We extensively showcase the scalability and performance of our proposal on several datasets with up to 8 million observations, and various DGP architectures with up to 30 hidden layers.