# Probabilistic Deep Learning with Probabilistic Neural Networks and Deep Probabilistic Models

arXiv: https://arxiv.org/abs/2106.00120 · HTML: https://mlsanity.com/paper/2106.00120/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 12 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 8 of 8)

- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [TensorFlow Distributions](https://mlsanity.com/paper/1711.10604/) (2017)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Bayesian Layers: A Module for Neural Network Uncertainty](https://mlsanity.com/paper/1812.03973/) (2018)
- [Predictive Uncertainty Quantification with Compound Density Networks](https://mlsanity.com/paper/1902.01080/) (2019)
- [Bayesian Hyperparameter Optimization with BoTorch, GPyTorch and Ax](https://mlsanity.com/paper/1912.05686/) (2019)
- [Joint Distributions for TensorFlow Probability](https://mlsanity.com/paper/2001.11819/) (2020)
- [Probabilistic Models with Deep Neural Networks](https://mlsanity.com/paper/1908.03442/) (2019)

## Built on by (most cited first, 4 of 4)

- [Bayesian Neural Networks: Essentials](https://mlsanity.com/paper/2106.13594/) (2021)
- [An Uncertainty-aware Deep Learning Framework-based Robust Design Optimization of Metamaterial Units](https://mlsanity.com/paper/2407.20251/) (2024)
- [Hybrid Bayesian Neural Networks with Functional Probabilistic Layers](https://mlsanity.com/paper/2107.07014/) (2021)
- [A Deep Learning Technique to Control the Non-linear Dynamics of a Gravitational-wave Interferometer](https://mlsanity.com/paper/2302.07921/) (2023)

## Abstract (arXiv)

Probabilistic deep learning is deep learning that accounts for uncertainty, both model uncertainty and data uncertainty. It is based on the use of probabilistic models and deep neural networks. We distinguish two approaches to probabilistic deep learning: probabilistic neural networks and deep probabilistic models. The former employs deep neural networks that utilize probabilistic layers which can represent and process uncertainty; the latter uses probabilistic models that incorporate deep neural network components which capture complex non-linear stochastic relationships between the random variables. We discuss some major examples of each approach including Bayesian neural networks and mixture density networks (for probabilistic neural networks), and variational autoencoders, deep Gaussian processes and deep mixed effects models (for deep probabilistic models). TensorFlow Probability is a library for probabilistic modeling and inference which can be used for both approaches of probabilistic deep learning. We include its code examples for illustration.