# Understanding predictive information criteria for Bayesian models

arXiv: https://arxiv.org/abs/1307.5928 · HTML: https://mlsanity.com/paper/1307.5928/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2013.

## Built on by (most cited first, 30 of 52)

- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetry](https://mlsanity.com/paper/2304.02902/) (2023)
- [Sensitivity-Aware Amortized Bayesian Inference](https://mlsanity.com/paper/2310.11122/) (2023)
- [Bayesian Active Learning for Scanning Probe Microscopy: from Gaussian Processes to Hypothesis Learning](https://mlsanity.com/paper/2205.15458/) (2022)
- [Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learning](https://mlsanity.com/paper/2507.05526/) (2025)
- [Microcanonical Langevin Ensembles: Advancing the Sampling of Bayesian Neural Networks](https://mlsanity.com/paper/2502.06335/) (2025)
- [Are you using test log-likelihood correctly?](https://mlsanity.com/paper/2212.00219/) (2022)
- [Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes](https://mlsanity.com/paper/2111.04558/) (2021)
- [Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?](https://mlsanity.com/paper/2602.06500/) (2026)
- [An Ordinal Latent Variable Model of Conflict Intensity](https://mlsanity.com/paper/2210.03971/) (2022)
- [Paths and Ambient Spaces in Neural Loss Landscapes](https://mlsanity.com/paper/2503.03382/) (2025)
- [Mathematical Theory of Bayesian Statistics for Unknown Information Source](https://mlsanity.com/paper/2206.05630/) (2022)
- [On Masked Pre-training and the Marginal Likelihood](https://mlsanity.com/paper/2306.00520/) (2023)
- [A Review and Roadmap of Deep Learning Causal Discovery in Different Variable Paradigms](https://mlsanity.com/paper/2209.06367/) (2022)
- [Recent Advances in Algebraic Geometry and Bayesian Statistics](https://mlsanity.com/paper/2211.10049/) (2022)
- [Gradient-free variational learning with conditional mixture networks](https://mlsanity.com/paper/2408.16429/) (2024)
- [Efficient and Accurate Learning of Mixtures of Plackett-Luce Models](https://mlsanity.com/paper/2302.05343/) (2023)
- [SEAM: An Integrated Activation-Coupled Model of Sentence Processing and Eye Movements in Reading](https://mlsanity.com/paper/2303.05221/) (2023)
- [On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors](https://mlsanity.com/paper/2603.22030/) (2026)
- [A Latent Space Model for Multilayer Network Data](https://mlsanity.com/paper/2102.09560/) (2021)
- [FIFA ranking: Evaluation and path forward](https://mlsanity.com/paper/2201.00691/) (2021)
- [A Differentially Private Probabilistic Framework for Modeling the Variability Across Federated Datasets of Heterogeneous Multi-View Observations](https://mlsanity.com/paper/2204.07352/) (2022)
- [The Ordered Matrix Dirichlet for State-Space Models](https://mlsanity.com/paper/2212.04130/) (2022)
- [Bayesian CART models for insurance claims frequency](https://mlsanity.com/paper/2303.01923/) (2023)
- [Sparsifying Bayesian neural networks with latent binary variables and normalizing flows](https://mlsanity.com/paper/2305.03395/) (2023)
- [Discriminative calibration: Check Bayesian computation from simulations and flexible classifier](https://mlsanity.com/paper/2305.14593/) (2023)
- [Review and Prospect of Algebraic Research in Equivalent Framework between Statistical Mechanics and Machine Learning Theory](https://mlsanity.com/paper/2406.10234/) (2024)
- [Empowering Bayesian Neural Networks with Functional Priors through Anchored Ensembling for Mechanics Surrogate Modeling Applications](https://mlsanity.com/paper/2409.05234/) (2024)
- [Incorporating Expert Knowledge into Bayesian Causal Discovery of Mixtures of Directed Acyclic Graphs](https://mlsanity.com/paper/2510.06735/) (2025)
- [Recovering Latent Structures after Variational Bayesian Variable Selection: Fit Assessment and Factor-Number Selection in Partially Exploratory Factor Analysis](https://mlsanity.com/paper/2607.07159/) (2026)

## Abstract (arXiv)

We review the Akaike, deviance, and Watanabe-Akaike information criteria from a Bayesian perspective, where the goal is to estimate expected out-of-sample-prediction error using a biascorrected adjustment of within-sample error. We focus on the choices involved in setting up these measures, and we compare them in three simple examples, one theoretical and two applied. The contribution of this review is to put all these information criteria into a Bayesian predictive context and to better understand, through small examples, how these methods can apply in practice.