# Stabilising priors for robust Bayesian deep learning

arXiv: https://arxiv.org/abs/1910.10386 · HTML: https://mlsanity.com/paper/1910.10386/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 2 of 2)

- [Graph Reparameterizations for Enabling 1000+ Monte Carlo Iterations in Bayesian Deep Neural Networks](https://mlsanity.com/paper/2202.09478/) (2022)
- [Radial Spike and Slab Bayesian Neural Networks for Sparse Data in Ransomware Attacks](https://mlsanity.com/paper/2205.14759/) (2022)

## Abstract (arXiv)

Bayesian neural networks (BNNs) have developed into useful tools for probabilistic modelling due to recent advances in variational inference enabling large scale BNNs. However, BNNs remain brittle and hard to train, especially: (1) when using deep architectures consisting of many hidden layers and (2) in situations with large weight variances. We use signal propagation theory to quantify these challenges and propose self-stabilising priors. This is achieved by a reformulation of the ELBO to allow the prior to influence network signal propagation. Then, we develop a stabilising prior, where the distributional parameters of the prior are adjusted before each forward pass to ensure stability of the propagating signal. This stabilised signal propagation leads to improved convergence and robustness making it possible to train deeper networks and in more noisy settings.