# Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning

arXiv: https://arxiv.org/abs/2012.08101 · HTML: https://mlsanity.com/paper/2012.08101/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2020.

## Built on by (most cited first, 18 of 18)

- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [Koopman Neural Forecaster for Time Series with Temporal Distribution Shifts](https://mlsanity.com/paper/2210.03675/) (2022)
- [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://mlsanity.com/paper/2402.14989/) (2024)
- [Active learning for data streams: a survey](https://mlsanity.com/paper/2302.08893/) (2023)
- [Variational Control for Guidance in Diffusion Models](https://mlsanity.com/paper/2502.03686/) (2025)
- [Active Test-Time Adaptation: Theoretical Analyses and An Algorithm](https://mlsanity.com/paper/2404.05094/) (2024)
- [Kalman Filter for Online Classification of Non-Stationary Data](https://mlsanity.com/paper/2306.08448/) (2023)
- [When to retrain a machine learning model](https://mlsanity.com/paper/2505.14903/) (2025)
- [Memory-free Online Change-point Detection: A Novel Neural Network Approach](https://mlsanity.com/paper/2207.03932/) (2022)
- [Modyn: Data-Centric Machine Learning Pipeline Orchestration](https://mlsanity.com/paper/2312.06254/) (2023)
- [A unifying framework for generalised Bayesian online learning in non-stationary environments](https://mlsanity.com/paper/2411.10153/) (2024)
- [BAM: Bayes with Adaptive Memory](https://mlsanity.com/paper/2202.02405/) (2022)
- [RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget](https://mlsanity.com/paper/2505.24149/) (2025)
- [Zero-Shot Anomaly Detection via Batch Normalization](https://mlsanity.com/paper/2302.07849/) (2023)
- [Probabilistic Hash Embeddings for Online Learning of Categorical Features](https://mlsanity.com/paper/2511.20893/) (2025)
- [Learn to Bid as a Price-Maker Wind Power Producer](https://mlsanity.com/paper/2503.16107/) (2025)
- [Protected Probabilistic Classification Library](https://mlsanity.com/paper/2509.11267/) (2025)
- [Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Direction](https://mlsanity.com/paper/2510.15422/) (2025)

## Abstract (arXiv)

We consider the problem of online learning in the presence of distribution shifts that occur at an unknown rate and of unknown intensity. We derive a new Bayesian online inference approach to simultaneously infer these distribution shifts and adapt the model to the detected changes by integrating ideas from change point detection, switching dynamical systems, and Bayesian online learning. Using a binary 'change variable,' we construct an informative prior such that--if a change is detected--the model partially erases the information of past model updates by tempering to facilitate adaptation to the new data distribution. Furthermore, the approach uses beam search to track multiple change-point hypotheses and selects the most probable one in hindsight. Our proposed method is model-agnostic, applicable in both supervised and unsupervised learning settings, suitable for an environment of concept drifts or covariate drifts, and yields improvements over state-of-the-art Bayesian online learning approaches.