# A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference

arXiv: https://arxiv.org/abs/1901.02731 · HTML: https://mlsanity.com/paper/1901.02731/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2019.

## Built on by (most cited first, 30 of 51)

- [Bayesian Low-rank Adaptation for Large Language Models](https://mlsanity.com/paper/2308.13111/) (2023)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice](https://mlsanity.com/paper/2410.15326/) (2024)
- [Uncertainty-based Fairness Measures](https://mlsanity.com/paper/2312.11299/) (2023)
- [Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Tasks](https://mlsanity.com/paper/2505.20047/) (2025)
- [Bayesian neural networks via MCMC: a Python-based tutorial](https://mlsanity.com/paper/2304.02595/) (2023)
- [Reducing Overconfidence Predictions for Autonomous Driving Perception](https://mlsanity.com/paper/2202.07825/) (2022)
- [Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability](https://mlsanity.com/paper/2107.08461/) (2021)
- [Modeling Annotation Uncertainty with Gaussian Heatmaps in Landmark Localization](https://mlsanity.com/paper/2109.09533/) (2021)
- [Wavelength-aware 2D Convolutions for Hyperspectral Imaging](https://mlsanity.com/paper/2209.03136/) (2022)
- [Bayesian neural networks for predicting uncertainty in full-field material response](https://mlsanity.com/paper/2406.14838/) (2024)
- [An Uncertainty-Informed Framework for Trustworthy Fault Diagnosis in Safety-Critical Applications](https://mlsanity.com/paper/2111.00874/) (2021)
- [X-Fake: Juggling Utility Evaluation and Explanation of Simulated SAR Images](https://mlsanity.com/paper/2407.19436/) (2024)
- [TapType: Ten-finger text entry on everyday surfaces via Bayesian inference](https://mlsanity.com/paper/2410.06001/) (2024)
- [Delusions of Large Language Models](https://mlsanity.com/paper/2503.06709/) (2025)
- [Recurrent Localization Networks applied to the Lippmann-Schwinger Equation](https://mlsanity.com/paper/2102.00063/) (2021)
- [W2WNet: a two-module probabilistic Convolutional Neural Network with embedded data cleansing functionality](https://mlsanity.com/paper/2103.13107/) (2021)
- [Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving](https://mlsanity.com/paper/2110.03553/) (2021)
- [A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel](https://mlsanity.com/paper/2111.10722/) (2021)
- [EffLoc: Lightweight Vision Transformer for Efficient 6-DOF Camera Relocalization](https://mlsanity.com/paper/2402.13537/) (2024)
- [Distance-informed Neural Processes](https://mlsanity.com/paper/2508.18903/) (2025)
- [Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System](https://mlsanity.com/paper/2107.13252/) (2021)
- [Bayesian Convolutional Neural Networks for Limited Data Hyperspectral Remote Sensing Image Classification](https://mlsanity.com/paper/2205.09250/) (2022)
- [Uncertainty-aware U-Net for Medical Landmark Detection](https://mlsanity.com/paper/2303.10349/) (2023)
- [A Survey and Analysis on Automated Glioma Brain Tumor Segmentation and Overall Patient Survival Prediction](https://mlsanity.com/paper/2101.10599/) (2021)
- [A Bayesian Approach to Invariant Deep Neural Networks](https://mlsanity.com/paper/2107.09301/) (2021)
- [Cluster-based Regression using Variational Inference and Applications in Financial Forecasting](https://mlsanity.com/paper/2205.00605/) (2022)
- [Latent Preserving Generative Adversarial Network for Imbalance classification](https://mlsanity.com/paper/2209.01555/) (2022)
- [SACDNet: Towards Early Type 2 Diabetes Prediction with Uncertainty for Electronic Health Records](https://mlsanity.com/paper/2301.04844/) (2023)
- [Hierarchical Prompts for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2401.11544/) (2024)

## Abstract (arXiv)

Artificial Neural Networks are connectionist systems that perform a given task by learning on examples without having prior knowledge about the task. This is done by finding an optimal point estimate for the weights in every node. Generally, the network using point estimates as weights perform well with large datasets, but they fail to express uncertainty in regions with little or no data, leading to overconfident decisions. In this paper, Bayesian Convolutional Neural Network (BayesCNN) using Variational Inference is proposed, that introduces probability distribution over the weights. Furthermore, the proposed BayesCNN architecture is applied to tasks like Image Classification, Image Super-Resolution and Generative Adversarial Networks. The results are compared to point-estimates based architectures on MNIST, CIFAR-10 and CIFAR-100 datasets for Image CLassification task, on BSD300 dataset for Image Super Resolution task and on CIFAR10 dataset again for Generative Adversarial Network task. BayesCNN is based on Bayes by Backprop which derives a variational approximation to the true posterior. We, therefore, introduce the idea of applying two convolutional operations, one for the mean and one for the variance. Our proposed method not only achieves performances equivalent to frequentist inference in identical architectures but also incorporate a measurement for uncertainties and regularisation. It further eliminates the use of dropout in the model. Moreover, we predict how certain the model prediction is based on the epistemic and aleatoric uncertainties and empirically show how the uncertainty can decrease, allowing the decisions made by the network to become more deterministic as the training accuracy increases. Finally, we propose ways to prune the Bayesian architecture and to make it more computational and time effective.