# Adaptive Bayesian Online Learning via Expert Aggregation

arXiv: https://arxiv.org/abs/2607.20239 · HTML: https://mlsanity.com/paper/2607.20239/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 19 of 19)

- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Adaptive Conformal Predictions for Time Series](https://mlsanity.com/paper/2202.07282/) (2022)
- [Conformal Inference for Online Prediction with Arbitrary Distribution Shifts](https://mlsanity.com/paper/2208.08401/) (2022)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Improved Online Conformal Prediction via Strongly Adaptive Online Learning](https://mlsanity.com/paper/2302.07869/) (2023)
- [Improved Strongly Adaptive Online Learning using Coin Betting](https://mlsanity.com/paper/1610.04578/) (2016)
- [Streaming Variational Bayes](https://mlsanity.com/paper/1307.6769/) (2013)
- [Adaptive Regret for Control of Time-Varying Dynamics](https://mlsanity.com/paper/2007.04393/) (2020)
- [Provable Bayesian Inference via Particle Mirror Descent](https://mlsanity.com/paper/1506.03101/) (2015)
- [A comparison of learning rate selection methods in generalized Bayesian inference](https://mlsanity.com/paper/2012.11349/) (2020)
- [Online PAC-Bayes Learning](https://mlsanity.com/paper/2206.00024/) (2022)
- [Non-exponentially weighted aggregation: regret bounds for unbounded loss functions](https://mlsanity.com/paper/2009.03017/) (2020)
- [A Generalization Bound for Online Variational Inference](https://mlsanity.com/paper/1904.03920/) (2019)
- [AMF: Aggregated Mondrian Forests for Online Learning](https://mlsanity.com/paper/1906.10529/) (2019)
- [Dual Adaptivity: A Universal Algorithm for Minimizing the Adaptive Regret of Convex Functions](https://mlsanity.com/paper/1906.10851/) (2019)
- [MARS via LASSO](https://mlsanity.com/paper/2111.11694/) (2021)
- [Sequential Gibbs Posteriors with Applications to Principal Component Analysis](https://mlsanity.com/paper/2310.12882/) (2023)
- [Online conformal inference with retrospective adjustment for faster adaptation to distribution shift](https://mlsanity.com/paper/2511.04275/) (2025)

## Built on by (most cited first, 1 of 1)

- [Fast rates in Bayesian online learning with approximate posteriors](https://mlsanity.com/paper/2608.25706/) (2026)

## Abstract (arXiv)

Bayesian online learning promises uncertainty-aware prediction on data streams, but its performance hinges on inferential choices, including learning rates, prior distributions and variational families, which are usually fixed before seeing the stream. We address this by treating Bayesian update rules as experts and aggregating the Bayesian experts according to sequential predictive losses. We prove that the resulting aggregate competes with the best expert in hindsight at an aggregation cost determined by how each expert's per-round performance is evaluated. We instantiate the framework in online conformal inference and Gaussian process regression. The conformal inference application yields a smoothed Bayesian counterpart of adaptive conformal inference with long-run randomized coverage, while the Gaussian process application gives an oracle inequality in cumulative predictive Kullback-Leibler risk and adaptation to unknown H"older smoothness up to logarithmic factors. Experiments show that the aggregate tracks strong experts without oracle expert selection.