# Deep Learning: A Bayesian Perspective

arXiv: https://arxiv.org/abs/1706.00473 · HTML: https://mlsanity.com/paper/1706.00473/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2017.

## Built on by (most cited first, 20 of 20)

- [Uncertainty-Aware Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2202.13071/) (2022)
- [Efficient Online Bayesian Inference for Neural Bandits](https://mlsanity.com/paper/2112.00195/) (2021)
- [Revisiting Bayesian Autoencoders with MCMC](https://mlsanity.com/paper/2104.05915/) (2021)
- [Bayesian graph convolutional neural networks via tempered MCMC](https://mlsanity.com/paper/2104.08438/) (2021)
- [Batch Bayesian optimisation via density-ratio estimation with guarantees](https://mlsanity.com/paper/2209.10715/) (2022)
- [Bayesian Neural Networks for Virtual Flow Metering: An Empirical Study](https://mlsanity.com/paper/2102.01391/) (2021)
- [Bayesian Deep Learning with Multilevel Trace-class Neural Networks](https://mlsanity.com/paper/2203.12961/) (2022)
- [How to Combine Variational Bayesian Networks in Federated Learning](https://mlsanity.com/paper/2206.10897/) (2022)
- [Merging Two Cultures: Deep and Statistical Learning](https://mlsanity.com/paper/2110.11561/) (2021)
- [Practical multi-fidelity machine learning: fusion of deterministic and Bayesian models](https://mlsanity.com/paper/2407.15110/) (2024)
- [Accelerating Langevin Monte Carlo Sampling: A Large Deviations Analysis](https://mlsanity.com/paper/2503.19066/) (2025)
- [Marginally calibrated response distributions for end-to-end learning in autonomous driving](https://mlsanity.com/paper/2110.01050/) (2021)
- [Low-rank variational Bayes correction to the Laplace method](https://mlsanity.com/paper/2111.12945/) (2021)
- [Decentralized Proximal Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2605.00723/) (2026)
- [Object Recognition in Different Lighting Conditions at Various Angles by Deep Learning Method](https://mlsanity.com/paper/2210.09618/) (2022)
- [Fusing Climate Data Products using a Spatially Varying Autoencoder](https://mlsanity.com/paper/2403.07822/) (2024)
- [Sparse Deep Learning Models with the ℓ1 Regularization](https://mlsanity.com/paper/2408.02801/) (2024)
- [Kolmogorov GAM Networks are all you need!](https://mlsanity.com/paper/2501.00704/) (2025)
- [Bayesian Double Descent](https://mlsanity.com/paper/2507.07338/) (2025)
- [Deep Neural Networks for Semiparametric Frailty Models via H-likelihood](https://mlsanity.com/paper/2307.06581/) (2023)

## Abstract (arXiv)

Deep learning is a form of machine learning for nonlinear high dimensional pattern matching and prediction. By taking a Bayesian probabilistic perspective, we provide a number of insights into more efficient algorithms for optimisation and hyper-parameter tuning. Traditional high-dimensional data reduction techniques, such as principal component analysis (PCA), partial least squares (PLS), reduced rank regression (RRR), projection pursuit regression (PPR) are all shown to be shallow learners. Their deep learning counterparts exploit multiple deep layers of data reduction which provide predictive performance gains. Stochastic gradient descent (SGD) training optimisation and Dropout (DO) regularization provide estimation and variable selection. Bayesian regularization is central to finding weights and connections in networks to optimize the predictive bias-variance trade-off. To illustrate our methodology, we provide an analysis of international bookings on Airbnb. Finally, we conclude with directions for future research.