# Bayesian Inference for Large Scale Image Classification

arXiv: https://arxiv.org/abs/1908.03491 · HTML: https://mlsanity.com/paper/1908.03491/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2019.

## Built on by (most cited first, 18 of 18)

- [Bayesian Neural Network Priors Revisited](https://mlsanity.com/paper/2102.06571/) (2021)
- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Quantifying Uncertainty in Deep Spatiotemporal Forecasting](https://mlsanity.com/paper/2105.11982/) (2021)
- [On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification](https://mlsanity.com/paper/2203.16481/) (2022)
- [Structured Stochastic Gradient MCMC](https://mlsanity.com/paper/2107.09028/) (2021)
- [Low-Precision Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2206.09909/) (2022)
- [DeepGLEAM: A hybrid mechanistic and deep learning model for COVID-19 forecasting](https://mlsanity.com/paper/2102.06684/) (2021)
- [Scalable Bayesian Uncertainty Quantification for Neural Network Potentials: Promise and Pitfalls](https://mlsanity.com/paper/2212.07959/) (2022)
- [Posterior concentrations of fully-connected Bayesian neural networks with general priors on the weights](https://mlsanity.com/paper/2403.14225/) (2024)
- [Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise](https://mlsanity.com/paper/2103.08497/) (2021)
- [On the optimization and pruning for Bayesian deep learning](https://mlsanity.com/paper/2210.12957/) (2022)
- [AdamMCMC: Combining Metropolis Adjusted Langevin with Momentum-based Optimization](https://mlsanity.com/paper/2312.14027/) (2023)
- [Stochastic Weight Sharing for Bayesian Neural Networks](https://mlsanity.com/paper/2505.17856/) (2025)
- [Masked Bayesian Neural Networks : Computation and Optimality](https://mlsanity.com/paper/2206.00853/) (2022)
- [A deep mixture density network for outlier-corrected interpolation of crowd-sourced weather data](https://mlsanity.com/paper/2201.10544/) (2022)
- [Piecewise Deterministic Markov Processes for Bayesian Neural Networks](https://mlsanity.com/paper/2302.08724/) (2023)
- [Addressing the Inconsistency in Bayesian Deep Learning via Generalized Laplace Approximation](https://mlsanity.com/paper/2405.13535/) (2024)
- [A Context-Aware Dual-Metric Framework for Confidence Estimation in Large Language Models](https://mlsanity.com/paper/2508.00600/) (2025)

## Abstract (arXiv)

Bayesian inference promises to ground and improve the performance of deep neural networks. It promises to be robust to overfitting, to simplify the training procedure and the space of hyperparameters, and to provide a calibrated measure of uncertainty that can enhance decision making, agent exploration and prediction fairness. Markov Chain Monte Carlo (MCMC) methods enable Bayesian inference by generating samples from the posterior distribution over model parameters. Despite the theoretical advantages of Bayesian inference and the similarity between MCMC and optimization methods, the performance of sampling methods has so far lagged behind optimization methods for large scale deep learning tasks. We aim to fill this gap and introduce ATMC, an adaptive noise MCMC algorithm that estimates and is able to sample from the posterior of a neural network. ATMC dynamically adjusts the amount of momentum and noise applied to each parameter update in order to compensate for the use of stochastic gradients. We use a ResNet architecture without batch normalization to test ATMC on the Cifar10 benchmark and the large scale ImageNet benchmark and show that, despite the absence of batch normalization, ATMC outperforms a strong optimization baseline in terms of both classification accuracy and test log-likelihood. We show that ATMC is intrinsically robust to overfitting on the training data and that ATMC provides a better calibrated measure of uncertainty compared to the optimization baseline.