# A comparison of learning rate selection methods in generalized Bayesian inference

arXiv: https://arxiv.org/abs/2012.11349 · HTML: https://mlsanity.com/paper/2012.11349/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2020.

## Built on by (most cited first, 23 of 23)

- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [Robust Bayesian Inference for Simulator-based Models via the MMD Posterior Bootstrap](https://mlsanity.com/paper/2202.04744/) (2022)
- [A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods](https://mlsanity.com/paper/2305.15027/) (2023)
- [Generalized Bayesian Inference for Scientific Simulators via Amortized Cost Estimation](https://mlsanity.com/paper/2305.15208/) (2023)
- [Robust and Scalable Bayesian Online Changepoint Detection](https://mlsanity.com/paper/2302.04759/) (2023)
- [Robust and Conjugate Gaussian Process Regression](https://mlsanity.com/paper/2311.00463/) (2023)
- [Robust and Conjugate Spatio-Temporal Gaussian Processes](https://mlsanity.com/paper/2502.02450/) (2025)
- [TabMGP: Martingale Posterior with TabPFN](https://mlsanity.com/paper/2510.25154/) (2025)
- [Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix](https://mlsanity.com/paper/2406.14269/) (2024)
- [Concentration properties of fractional posterior in 1-bit matrix completion](https://mlsanity.com/paper/2404.08969/) (2024)
- [Semiparametric inference using fractional posteriors](https://mlsanity.com/paper/2301.08158/) (2023)
- [Scalable Semi-Modular Inference with Variational Meta-Posteriors](https://mlsanity.com/paper/2204.00296/) (2022)
- [Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach](https://mlsanity.com/paper/2504.20319/) (2025)
- [Handling bounded response in high dimensions: a Horseshoe prior Bayesian Beta regression approach](https://mlsanity.com/paper/2505.22211/) (2025)
- [Multi-Output Robust and Conjugate Gaussian Processes](https://mlsanity.com/paper/2510.26401/) (2025)
- [Valid Inference for Machine Learning Model Parameters](https://mlsanity.com/paper/2302.10840/) (2023)
- [Sequential Gibbs Posteriors with Applications to Principal Component Analysis](https://mlsanity.com/paper/2310.12882/) (2023)
- [Adaptive Bayesian Online Learning via Expert Aggregation](https://mlsanity.com/paper/2607.20239/) (2026)
- [Generalised Robust Bayes for Joint Inference of Model and Contamination](https://mlsanity.com/paper/2607.25665/) (2026)
- [BOCO: Bayesian Online Contextual Optimization for Decision-Focused Online Learning](https://mlsanity.com/paper/2511.20413/) (2025)
- [Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance](https://mlsanity.com/paper/2601.16425/) (2026)
- [General Bayesian Policy Learning](https://mlsanity.com/paper/2602.23672/) (2026)
- [Hybrid Adaptive Kalman Filtering for Data-Efficient Joint Tracking and Classification](https://mlsanity.com/paper/2606.02767/) (2026)

## Abstract (arXiv)

Generalized Bayes posterior distributions are formed by putting a fractional power on the likelihood before combining with the prior via Bayes's formula. This fractional power, which is often viewed as a remedy for potential model misspecification bias, is called the learning rate, and a number of data-driven learning rate selection methods have been proposed in the recent literature. Each of these proposals has a different focus, a different target they aim to achieve, which makes them difficult to compare. In this paper, we provide a direct head-to-head comparison of these learning rate selection methods in various misspecified model scenarios, in terms of several relevant metrics, in particular, coverage probability of the generalized Bayes credible regions. In some examples all the methods perform well, while in others the misspecification is too severe to be overcome, but we find that the so-called generalized posterior calibration algorithm tends to outperform the others in terms of credible region coverage probability.