# Are you using test log-likelihood correctly?

arXiv: https://arxiv.org/abs/2212.00219 · HTML: https://mlsanity.com/paper/2212.00219/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 43 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 23 of 23)

- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Validating Bayesian Inference Algorithms with Simulation-Based Calibration](https://mlsanity.com/paper/1804.06788/) (2018)
- [Subspace Inference for Bayesian Deep Learning](https://mlsanity.com/paper/1907.07504/) (2019)
- [Measuring Sample Quality with Stein's Method](https://mlsanity.com/paper/1506.03039/) (2015)
- [Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors](https://mlsanity.com/paper/1603.04733/) (2016)
- [Quality of Uncertainty Quantification for Bayesian Neural Network Inference](https://mlsanity.com/paper/1906.09686/) (2019)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [Understanding predictive information criteria for Bayesian models](https://mlsanity.com/paper/1307.5928/) (2013)
- [Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes](https://mlsanity.com/paper/2005.08140/) (2020)
- [Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors](https://mlsanity.com/paper/1806.05975/) (2018)
- [SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient](https://mlsanity.com/paper/1811.04504/) (2018)
- [Validated Variational Inference via Practical Posterior Error Bounds](https://mlsanity.com/paper/1910.04102/) (2019)
- [Visualization in Bayesian workflow](https://mlsanity.com/paper/1709.01449/) (2017)
- [Neural Variational Gradient Descent](https://mlsanity.com/paper/2107.10731/) (2021)
- [Scalable Bayesian Learning of Recurrent Neural Networks for Language Modeling](https://mlsanity.com/paper/1611.08034/) (2016)
- [Reproducible Model Selection Using Bagged Posteriors](https://mlsanity.com/paper/2007.14845/) (2020)
- [High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models](https://mlsanity.com/paper/1512.07662/) (2015)

## Built on by (most cited first, 8 of 8)

- [Thermometer: Towards Universal Calibration for Large Language Models](https://mlsanity.com/paper/2403.08819/) (2024)
- [Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?](https://mlsanity.com/paper/2602.06500/) (2026)
- [Practical multi-fidelity machine learning: fusion of deterministic and Bayesian models](https://mlsanity.com/paper/2407.15110/) (2024)
- [Variational Bayesian Neural Networks via Resolution of Singularities](https://mlsanity.com/paper/2302.06035/) (2023)
- [Temperature Optimization for Bayesian Deep Learning](https://mlsanity.com/paper/2410.05757/) (2024)
- [Understanding the Trade-offs in Accuracy and Uncertainty Quantification: Architecture and Inference Choices in Bayesian Neural Networks](https://mlsanity.com/paper/2503.11808/) (2025)
- [Pathwise Gradient Variance Reduction with Control Variates in Variational Inference](https://mlsanity.com/paper/2410.05753/) (2024)
- [Adaptive Bayesian Data-Driven Design of Reliable Solder Joints for Micro-electronic Devices](https://mlsanity.com/paper/2507.19663/) (2025)

## Abstract (arXiv)

Test log-likelihood is commonly used to compare different models of the same data or different approximate inference algorithms for fitting the same probabilistic model. We present simple examples demonstrating how comparisons based on test log-likelihood can contradict comparisons according to other objectives. Specifically, our examples show that (i) approximate Bayesian inference algorithms that attain higher test log-likelihoods need not also yield more accurate posterior approximations and (ii) conclusions about forecast accuracy based on test log-likelihood comparisons may not agree with conclusions based on root mean squared error.