# Posterior concentration and fast convergence rates for generalized Bayesian learning

arXiv: https://arxiv.org/abs/2111.10243 · HTML: https://mlsanity.com/paper/2111.10243/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 14 of 14)

- [Mean estimation and regression under heavy-tailed distributions--a survey](https://mlsanity.com/paper/1906.04280/) (2019)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [PAC-Bayesian Theory Meets Bayesian Inference](https://mlsanity.com/paper/1605.08636/) (2016)
- [Loss minimization and parameter estimation with heavy tails](https://mlsanity.com/paper/1307.1827/) (2013)
- [Learning without Concentration](https://mlsanity.com/paper/1401.0304/) (2014)
- [Fast rates in statistical and online learning](https://mlsanity.com/paper/1507.02592/) (2015)
- [Empirical risk minimization for heavy-tailed losses](https://mlsanity.com/paper/1406.2462/) (2014)
- [Safe-Bayesian Generalized Linear Regression](https://mlsanity.com/paper/1910.09227/) (2019)
- [On aggregation for heavy-tailed classes](https://mlsanity.com/paper/1502.07097/) (2015)
- [Generalization and Robustness of Batched Weighted Average Algorithm with V-geometrically Ergodic Markov Data](https://mlsanity.com/paper/1406.3166/) (2014)
- [Learning From Non-iid Data: Fast Rates for the One-vs-All Multiclass Plug-in Classifiers](https://mlsanity.com/paper/1408.2714/) (2014)
- [From Stochastic Mixability to Fast Rates](https://mlsanity.com/paper/1406.3781/) (2014)
- [Fast learning rates with heavy-tailed losses](https://mlsanity.com/paper/1609.09481/) (2016)
- [Borrowing strengh in hierarchical Bayes: Posterior concentration of the Dirichlet base measure](https://mlsanity.com/paper/1301.0802/) (2013)

## Built on by (most cited first, 2 of 2)

- [Sequential Importance Sampling for Hybrid Model Bayesian Inference to Support Bioprocess Mechanism Learning and Robust Control](https://mlsanity.com/paper/2205.02410/) (2022)
- [EnSolver: Uncertainty-Aware Ensemble CAPTCHA Solvers with Theoretical Guarantees](https://mlsanity.com/paper/2307.15180/) (2023)

## Abstract (arXiv)

In this paper, we study the learning rate of generalized Bayes estimators in a general setting where the hypothesis class can be uncountable and have an irregular shape, the loss function can have heavy tails, and the optimal hypothesis may not be unique. We prove that under the multi-scale Bernstein's condition, the generalized posterior distribution concentrates around the set of optimal hypotheses and the generalized Bayes estimator can achieve fast learning rate. Our results are applied to show that the standard Bayesian linear regression is robust to heavy-tailed distributions.