# Rethinking Likelihood distributions: Student's t Likelihood Boosts Bayesian Neural Network Performance

arXiv: https://arxiv.org/abs/2607.25376 · HTML: https://mlsanity.com/paper/2607.25376/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Builds on (most cited first, 17 of 17)

- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors](https://mlsanity.com/paper/2005.07186/) (2020)
- [Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam](https://mlsanity.com/paper/1806.04854/) (2018)
- [Bayesian Neural Network Priors Revisited](https://mlsanity.com/paper/2102.06571/) (2021)
- [Generalized Variational Inference: Three arguments for deriving new Posteriors](https://mlsanity.com/paper/1904.02063/) (2019)
- [Model Selection in Bayesian Neural Networks via Horseshoe Priors](https://mlsanity.com/paper/1705.10388/) (2017)
- [Student-t Processes as Alternatives to Gaussian Processes](https://mlsanity.com/paper/1402.4306/) (2014)
- [Precise characterization of the prior predictive distribution of deep ReLU networks](https://mlsanity.com/paper/2106.06615/) (2021)
- [The prior can generally only be understood in the context of the likelihood](https://mlsanity.com/paper/1708.07487/) (2017)
- [Variational Inference based on Robust Divergences](https://mlsanity.com/paper/1710.06595/) (2017)
- [Scalable Spatiotemporal Prediction with Bayesian Neural Fields](https://mlsanity.com/paper/2403.07657/) (2024)
- [Scale Mixtures of Neural Network Gaussian Processes](https://mlsanity.com/paper/2107.01408/) (2021)

## Abstract (arXiv)

In Bayesian neural networks (BNNs), variational inference is a widely adopted framework for modeling uncertainty in a distributional way, with the evidence lower bound (ELBO) serving as the standard objective function. Several distributions contribute to the ELBO loss, such as the prior, approximated posterior, and likelihood distribution. Typically, these distributions are all approximated by a Gaussian distribution, since it is easy to compute, allows for reparameterized gradients, and provides a closed-form loss for training. However, several works have highlighted that this assumption may not generally hold, posing the risk of model misspecification. Alternative distributions have been proposed for the prior specifically, while the effect of distribution choice on the likelihood distribution remains unexplored. In this work, our aim is to close this gap by investigating whether alternative assumptions for the likelihood distribution can outperform the commonly used Gaussian. We compare several likelihood distribution assumptions, such as skewed or heavy-tailed, across regression tasks on both artificial and real-world datasets using standard multilayer perceptrons (MLPs). Our findings demonstrate that Student's t yields better predictive performance than a Gaussian likelihood distribution, independent of the data distribution and MLP architecture (depth and width). In some cases, Student's t can also lead to shorter training times, while still being easy to implement.