# On Batch Normalisation for Approximate Bayesian Inference

arXiv: https://arxiv.org/abs/2012.13220 · HTML: https://mlsanity.com/paper/2012.13220/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Dec 2020.

## Built on by (most cited first, 3 of 3)

- [Controlled Dropout for Uncertainty Estimation](https://mlsanity.com/paper/2205.03109/) (2022)
- [Uncertainty quantification with approximate variational learning for wearable photoplethysmography prediction tasks](https://mlsanity.com/paper/2505.11412/) (2025)
- [BCN: Batch Channel Normalization for Image Classification](https://mlsanity.com/paper/2312.00596/) (2023)

## Abstract (arXiv)

We study batch normalisation in the context of variational inference methods in Bayesian neural networks, such as mean-field or MC Dropout. We show that batch-normalisation does not affect the optimum of the evidence lower bound (ELBO). Furthermore, we study the Monte Carlo Batch Normalisation (MCBN) algorithm, proposed as an approximate inference technique parallel to MC Dropout, and show that for larger batch sizes, MCBN fails to capture epistemic uncertainty. Finally, we provide insights into what is required to fix this failure, namely having to view the mini-batch size as a variational parameter in MCBN. We comment on the asymptotics of the ELBO with respect to this variational parameter, showing that as dataset size increases towards infinity, the batch-size must increase towards infinity as well for MCBN to be a valid approximate inference technique.