# Convolutional Normalizing Flows for Deep Gaussian Processes

arXiv: https://arxiv.org/abs/2104.08472 · HTML: https://mlsanity.com/paper/2104.08472/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 63 references that are in the map.
- First posted on arXiv 17 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [NIPS 2016 Tutorial: Generative Adversarial Networks](https://mlsanity.com/paper/1701.00160/) (2016)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [Federated Bayesian Optimization via Thompson Sampling](https://mlsanity.com/paper/2010.10154/) (2020)
- [Convolutional Gaussian Processes](https://mlsanity.com/paper/1709.01894/) (2017)
- [Deep Gaussian Processes for Regression using Approximate Expectation Propagation](https://mlsanity.com/paper/1602.04133/) (2016)
- [Random Feature Expansions for Deep Gaussian Processes](https://mlsanity.com/paper/1610.04386/) (2016)
- [Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1806.05490/) (2018)
- [Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds](https://mlsanity.com/paper/1810.12263/) (2018)
- [Decentralized High-Dimensional Bayesian Optimization with Factor Graphs](https://mlsanity.com/paper/1711.07033/) (2017)
- [Top-k Ranking Bayesian Optimization](https://mlsanity.com/paper/2012.10688/) (2020)
- [Variational Auto-encoded Deep Gaussian Processes](https://mlsanity.com/paper/1511.06455/) (2015)
- [Implicit Posterior Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1910.11998/) (2019)
- [Gaussian Process Planning with Lipschitz Continuous Reward Functions: Towards Unifying Bayesian Optimization, Active Learning, and Beyond](https://mlsanity.com/paper/1511.06890/) (2015)
- [Bayesian Optimization with Binary Auxiliary Information](https://mlsanity.com/paper/1906.07277/) (2019)
- [Private Outsourced Bayesian Optimization](https://mlsanity.com/paper/2010.12799/) (2020)
- [An Information-Theoretic Framework for Unifying Active Learning Problems](https://mlsanity.com/paper/2012.10695/) (2020)
- [R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games](https://mlsanity.com/paper/2006.16679/) (2020)
- [Near-Optimal Active Learning of Multi-Output Gaussian Processes](https://mlsanity.com/paper/1511.06891/) (2015)
- [Nonmyopic Gaussian Process Optimization with Macro-Actions](https://mlsanity.com/paper/2002.09670/) (2020)
- [GP-Localize: Persistent Mobile Robot Localization using Online Sparse Gaussian Process Observation Model](https://mlsanity.com/paper/1404.5165/) (2014)
- [Nested Variational Compression in Deep Gaussian Processes](https://mlsanity.com/paper/1412.1370/) (2014)
- [Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression](https://mlsanity.com/paper/1912.02641/) (2019)
- [Multi-Robot Informative Path Planning for Active Sensing of Environmental Phenomena: A Tale of Two Algorithms](https://mlsanity.com/paper/1302.0723/) (2013)
- [Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations](https://mlsanity.com/paper/1305.5826/) (2013)
- [A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression](https://mlsanity.com/paper/1611.06080/) (2016)
- [Gaussian Process-Based Decentralized Data Fusion and Active Sensing for Mobility-on-Demand System](https://mlsanity.com/paper/1306.1491/) (2013)

## Built on by (most cited first, 2 of 2)

- [Bayesian Optimization under Stochastic Delayed Feedback](https://mlsanity.com/paper/2206.09341/) (2022)
- [Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes](https://mlsanity.com/paper/2309.12658/) (2023)

## Abstract (arXiv)

Deep Gaussian processes (DGPs), a hierarchical composition of GP models, have successfully boosted the expressive power of their single-layer counterpart. However, it is impossible to perform exact inference in DGPs, which has motivated the recent development of variational inference-based methods. Unfortunately, either these methods yield a biased posterior belief or it is difficult to evaluate their convergence. This paper introduces a new approach for specifying flexible, arbitrarily complex, and scalable approximate posterior distributions. The posterior distribution is constructed through a normalizing flow (NF) which transforms a simple initial probability into a more complex one through a sequence of invertible transformations. Moreover, a novel convolutional normalizing flow (CNF) is developed to improve the time efficiency and capture dependency between layers. Empirical evaluation shows that CNF DGP outperforms the state-of-the-art approximation methods for DGPs.