# Locking and Quacking: Stacking Bayesian model predictions by log-pooling and superposition

arXiv: https://arxiv.org/abs/2305.07334 · HTML: https://mlsanity.com/paper/2305.07334/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 12 May 2023.

## Builds on (most cited first, 7 of 7)

- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [Pareto Smoothed Importance Sampling](https://mlsanity.com/paper/1507.02646/) (2015)
- [Bayesian hierarchical stacking: Some models are (somewhere) useful](https://mlsanity.com/paper/2101.08954/) (2021)
- [General Bayesian Loss Function Selection and the use of Improper Models](https://mlsanity.com/paper/2106.01214/) (2021)
- [Objective Bayesian inference with proper scoring rules](https://mlsanity.com/paper/1711.10819/) (2017)
- [Bayesian model comparison with the Hyv"arinen score: computation and consistency](https://mlsanity.com/paper/1711.00136/) (2017)
- [Combining probability distributions: Extending the logarithmic pooling approach](https://mlsanity.com/paper/1502.04206/) (2015)

## Abstract (arXiv)

Combining predictions from different models is a central problem in Bayesian inference and machine learning more broadly. Currently, these predictive distributions are almost exclusively combined using linear mixtures such as Bayesian model averaging, Bayesian stacking, and mixture of experts. Such linear mixtures impose idiosyncrasies that might be undesirable for some applications, such as multi-modality. While there exist alternative strategies (e.g. geometric bridge or superposition), optimising their parameters usually involves computing an intractable normalising constant repeatedly. We present two novel Bayesian model combination tools. These are generalisations of model stacking, but combine posterior densities by log-linear pooling (locking) and quantum superposition (quacking). To optimise model weights while avoiding the burden of normalising constants, we investigate the Hyvarinen score of the combined posterior predictions. We demonstrate locking with an illustrative example and discuss its practical application with importance sampling.