# Mitigating the Effects of Non-Identifiability on Inference for Bayesian Neural Networks with Latent Variables

arXiv: https://arxiv.org/abs/1911.00569 · HTML: https://mlsanity.com/paper/1911.00569/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2019.

## Built on by (most cited first, 2 of 2)

- [ELBOing Stein: Variational Bayes with Stein Mixture Inference](https://mlsanity.com/paper/2410.22948/) (2024)
- [Improving Forecasts of Suicide Attempts for Patients with Little Data](https://mlsanity.com/paper/2511.18199/) (2025)

## Abstract (arXiv)

Bayesian Neural Networks with Latent Variables (BNN+LVs) capture predictive uncertainty by explicitly modeling model uncertainty (via priors on network weights) and environmental stochasticity (via a latent input noise variable). In this work, we first show that BNN+LV suffers from a serious form of non-identifiability: explanatory power can be transferred between the model parameters and latent variables while fitting the data equally well. We demonstrate that as a result, in the limit of infinite data, the posterior mode over the network weights and latent variables is asymptotically biased away from the ground-truth. Due to this asymptotic bias, traditional inference methods may in practice yield parameters that generalize poorly and misestimate uncertainty. Next, we develop a novel inference procedure that explicitly mitigates the effects of likelihood non-identifiability during training and yields high-quality predictions as well as uncertainty estimates. We demonstrate that our inference method improves upon benchmark methods across a range of synthetic and real data-sets.