# Bayesian neural networks via MCMC: a Python-based tutorial

arXiv: https://arxiv.org/abs/2304.02595 · HTML: https://mlsanity.com/paper/2304.02595/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 158 references that are in the map.
- First posted on arXiv 2 Apr 2023.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks](https://mlsanity.com/paper/1512.05287/) (2015)
- [Conditional Time Series Forecasting with Convolutional Neural Networks](https://mlsanity.com/paper/1703.04691/) (2017)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Medical Image Analysis using Convolutional Neural Networks: A Review](https://mlsanity.com/paper/1709.02250/) (2017)
- [Deep Learning for Physical Processes: Incorporating Prior Scientific Knowledge](https://mlsanity.com/paper/1711.07970/) (2017)
- [Rank-normalization, folding, and localization: An improved widehatR for assessing convergence of MCMC](https://mlsanity.com/paper/1903.08008/) (2019)
- [A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference](https://mlsanity.com/paper/1901.02731/) (2019)
- [Autoregressive Convolutional Neural Networks for Asynchronous Time Series](https://mlsanity.com/paper/1703.04122/) (2017)
- [Evaluation of deep learning models for multi-step ahead time series prediction](https://mlsanity.com/paper/2103.14250/) (2021)
- [Revisiting the Gelman-Rubin Diagnostic](https://mlsanity.com/paper/1812.09384/) (2018)
- [Langevin-gradient parallel tempering for Bayesian neural learning](https://mlsanity.com/paper/1811.04343/) (2018)
- [Revisiting Bayesian Autoencoders with MCMC](https://mlsanity.com/paper/2104.05915/) (2021)
- [Bayesian graph convolutional neural networks via tempered MCMC](https://mlsanity.com/paper/2104.08438/) (2021)
- [Deep Importance Sampling based on Regression for Model Inversion and Emulation](https://mlsanity.com/paper/2010.10346/) (2020)
- [Metropolis Sampling](https://mlsanity.com/paper/1704.04629/) (2017)
- [Multi-core parallel tempering Bayeslands for basin and landscape evolution](https://mlsanity.com/paper/1806.10939/) (2018)
- [Post-Processing of MCMC](https://mlsanity.com/paper/2103.16048/) (2021)
- [Scaling up Data Augmentation MCMC via Calibration](https://mlsanity.com/paper/1703.03123/) (2017)
- [Bayeslands: A Bayesian inference approach for parameter uncertainty quantification in Badlands](https://mlsanity.com/paper/1805.03696/) (2018)
- [Bayesreef: A Bayesian inference framework for modelling reef growth in response to environmental change and biological dynamics](https://mlsanity.com/paper/1808.02763/) (2018)
- [Efficiency and robustness in Monte Carlo sampling of 3-D geophysical inversions with Obsidian v0.1.2: Setting up for success](https://mlsanity.com/paper/1812.00318/) (2018)

## Built on by (most cited first, 8 of 8)

- [Uncertainty Quantification in Machine Learning for Biosignal Applications -- A Review](https://mlsanity.com/paper/2312.09454/) (2023)
- [Review of deep learning models for crypto price prediction: implementation and evaluation](https://mlsanity.com/paper/2405.11431/) (2024)
- [Bayes-CATSI: A variational Bayesian deep learning framework for medical time series data imputation](https://mlsanity.com/paper/2410.01847/) (2024)
- [Convolutional neural networks for mineral prospecting through alteration mapping with remote sensing data](https://mlsanity.com/paper/2502.18533/) (2025)
- [Position: There Is No Free Bayesian Uncertainty Quantification](https://mlsanity.com/paper/2506.03670/) (2025)
- [Extreme value forecasting using relevance-based data augmentation with deep learning models](https://mlsanity.com/paper/2510.02407/) (2025)
- [tBayes-MICE: A Bayesian Approach to Multiple Imputation for Time Series Data](https://mlsanity.com/paper/2603.27142/) (2026)
- [Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks](https://mlsanity.com/paper/2607.08406/) (2026)

## Abstract (arXiv)

Bayesian inference provides a methodology for parameter estimation and uncertainty quantification in machine learning and deep learning methods. Variational inference and Markov Chain Monte-Carlo (MCMC) sampling methods are used to implement Bayesian inference. In the past three decades, MCMC sampling methods have faced some challenges in being adapted to larger models (such as in deep learning) and big data problems. Advanced proposal distributions that incorporate gradients, such as a Langevin proposal distribution, provide a means to address some of the limitations of MCMC sampling for Bayesian neural networks. Furthermore, MCMC methods have typically been constrained to statisticians and currently not well-known among deep learning researchers. We present a tutorial for MCMC methods that covers simple Bayesian linear and logistic models, and Bayesian neural networks. The aim of this tutorial is to bridge the gap between theory and implementation via coding, given a general sparsity of libraries and tutorials to this end. This tutorial provides code in Python with data and instructions that enable their use and extension. We provide results for some benchmark problems showing the strengths and weaknesses of implementing the respective Bayesian models via MCMC. We highlight the challenges in sampling multi-modal posterior distributions for the case of Bayesian neural networks and the need for further improvement of convergence diagnosis methods.