# Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection

arXiv: https://arxiv.org/abs/2107.13304 · HTML: https://mlsanity.com/paper/2107.13304/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 28 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Likelihood Ratios for Out-of-Distribution Detection](https://mlsanity.com/paper/1906.02845/) (2019)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Quality of Uncertainty Quantification for Bayesian Neural Network Inference](https://mlsanity.com/paper/1906.09686/) (2019)
- [An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos](https://mlsanity.com/paper/1801.03149/) (2018)
- [The continuous Bernoulli: fixing a pervasive error in variational autoencoders](https://mlsanity.com/paper/1907.06845/) (2019)
- [Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/1912.05651/) (2019)

## Built on by (most cited first, 4 of 4)

- [Robust outlier detection by de-biasing VAE likelihoods](https://mlsanity.com/paper/2108.08760/) (2021)
- [Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection](https://mlsanity.com/paper/2202.12653/) (2022)
- [Coalitional Bayesian Autoencoders -- Towards explainable unsupervised deep learning](https://mlsanity.com/paper/2110.10038/) (2021)
- [MSS-PAE: Saving Autoencoder-based Outlier Detection from Unexpected Reconstruction](https://mlsanity.com/paper/2304.00709/) (2023)

## Abstract (arXiv)

After an autoencoder (AE) has learnt to reconstruct one dataset, it might be expected that the likelihood on an out-of-distribution (OOD) input would be low. This has been studied as an approach to detect OOD inputs. Recent work showed this intuitive approach can fail for the dataset pairs FashionMNIST vs MNIST. This paper suggests this is due to the use of Bernoulli likelihood and analyses why this is the case, proposing two fixes: 1) Compute the uncertainty of likelihood estimate by using a Bayesian version of the AE. 2) Use alternative distributions to model the likelihood.