# Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC

arXiv: https://arxiv.org/abs/1507.04544 · HTML: https://mlsanity.com/paper/1507.04544/

## Key facts

- Cited by 110 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2015.

## Built on by (most cited first, 30 of 110)

- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [Systematic Inequalities in Language Technology Performance across the World's Languages](https://mlsanity.com/paper/2110.06733/) (2021)
- [Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations](https://mlsanity.com/paper/2504.14150/) (2025)
- [JANA: Jointly Amortized Neural Approximation of Complex Bayesian Models](https://mlsanity.com/paper/2302.09125/) (2023)
- [Automated Statistical Model Discovery with Language Models](https://mlsanity.com/paper/2402.17879/) (2024)
- [A review of predictive uncertainty estimation with machine learning](https://mlsanity.com/paper/2209.08307/) (2022)
- [Particle algorithms for maximum likelihood training of latent variable models](https://mlsanity.com/paper/2204.12965/) (2022)
- [A Representative Study on Human Detection of Artificially Generated Media Across Countries](https://mlsanity.com/paper/2312.05976/) (2023)
- [Sensitivity-Aware Amortized Bayesian Inference](https://mlsanity.com/paper/2310.11122/) (2023)
- [The Levers of Political Persuasion with Conversational AI](https://mlsanity.com/paper/2507.13919/) (2025)
- [Bayesian hierarchical stacking: Some models are (somewhere) useful](https://mlsanity.com/paper/2101.08954/) (2021)
- [Amortized Bayesian Multilevel Models](https://mlsanity.com/paper/2408.13230/) (2024)
- [Modeling structure-building in the brain with CCG parsing and large language models](https://mlsanity.com/paper/2210.16147/) (2022)
- [Generalization Through The Lens Of Leave-One-Out Error](https://mlsanity.com/paper/2203.03443/) (2022)
- [A Bayesian Bradley-Terry model to compare multiple ML algorithms on multiple data sets](https://mlsanity.com/paper/2208.04935/) (2022)
- [Capturing Humans' Mental Models of AI: An Item Response Theory Approach](https://mlsanity.com/paper/2305.09064/) (2023)
- [Proximal Interacting Particle Langevin Algorithms](https://mlsanity.com/paper/2406.14292/) (2024)
- [Model-based metrics: Sample-efficient estimates of predictive model subpopulation performance](https://mlsanity.com/paper/2104.12231/) (2021)
- [Systematic Training and Testing for Machine Learning Using Combinatorial Interaction Testing](https://mlsanity.com/paper/2201.12428/) (2022)
- [Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology](https://mlsanity.com/paper/2602.16703/) (2026)
- [Mathematical Theory of Bayesian Statistics for Unknown Information Source](https://mlsanity.com/paper/2206.05630/) (2022)
- [Learning Choice Functions with Gaussian Processes](https://mlsanity.com/paper/2302.00406/) (2023)
- [CriticAL: Critic Automation with Language Models](https://mlsanity.com/paper/2411.06590/) (2024)
- [RefineStat: Efficient Exploration for Probabilistic Program Synthesis](https://mlsanity.com/paper/2509.01082/) (2025)
- [Bayesian community detection for networks with covariates](https://mlsanity.com/paper/2203.02090/) (2022)
- [On the safe use of prior densities for Bayesian model selection](https://mlsanity.com/paper/2206.05210/) (2022)
- [Recent Advances in Algebraic Geometry and Bayesian Statistics](https://mlsanity.com/paper/2211.10049/) (2022)
- [Uncertainty Quantification and Propagation in Surrogate-based Bayesian Inference](https://mlsanity.com/paper/2312.05153/) (2023)
- [Bayesian calibration of stochastic agent based model via random forest](https://mlsanity.com/paper/2406.19524/) (2024)
- [Gradient-free variational learning with conditional mixture networks](https://mlsanity.com/paper/2408.16429/) (2024)

## Abstract (arXiv)

Leave-one-out cross-validation (LOO) and the widely applicable information criterion (WAIC) are methods for estimating pointwise out-of-sample prediction accuracy from a fitted Bayesian model using the log-likelihood evaluated at the posterior simulations of the parameter values. LOO and WAIC have various advantages over simpler estimates of predictive error such as AIC and DIC but are less used in practice because they involve additional computational steps. Here we lay out fast and stable computations for LOO and WAIC that can be performed using existing simulation draws. We introduce an efficient computation of LOO using Pareto-smoothed importance sampling (PSIS), a new procedure for regularizing importance weights. Although WAIC is asymptotically equal to LOO, we demonstrate that PSIS-LOO is more robust in the finite case with weak priors or influential observations. As a byproduct of our calculations, we also obtain approximate standard errors for estimated predictive errors and for comparing of predictive errors between two models. We implement the computations in an R package called 'loo' and demonstrate using models fit with the Bayesian inference package Stan.