# Closed Form Variational Objectives For Bayesian Neural Networks with a Single Hidden Layer

arXiv: https://arxiv.org/abs/1811.00686 · HTML: https://mlsanity.com/paper/1811.00686/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2018.

## Built on by (most cited first, 1 of 1)

- [Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise](https://mlsanity.com/paper/2103.08497/) (2021)

## Abstract (arXiv)

In this note we consider setups in which variational objectives for Bayesian neural networks can be computed in closed form. In particular we focus on single-layer networks in which the activation function is piecewise polynomial (e.g. ReLU). In this case we show that for a Normal likelihood and structured Normal variational distributions one can compute a variational lower bound in closed form. In addition we compute the predictive mean and variance in closed form. Finally, we also show how to compute approximate lower bounds for other likelihoods (e.g. softmax classification). In experiments we show how the resulting variational objectives can help improve training and provide fast test time predictions.