# A Unifying Bayesian View of Continual Learning

arXiv: https://arxiv.org/abs/1902.06494 · HTML: https://mlsanity.com/paper/1902.06494/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2019.

## Built on by (most cited first, 30 of 38)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Avalanche: an End-to-End Library for Continual Learning](https://mlsanity.com/paper/2104.00405/) (2021)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Do Bayesian Neural Networks Need To Be Fully Stochastic?](https://mlsanity.com/paper/2211.06291/) (2022)
- [Model Zoo: A Growing "Brain" That Learns Continually](https://mlsanity.com/paper/2106.03027/) (2021)
- [Posterior Meta-Replay for Continual Learning](https://mlsanity.com/paper/2103.01133/) (2021)
- [Class-Incremental Learning with Generative Classifiers](https://mlsanity.com/paper/2104.10093/) (2021)
- [Continual Learning for Recurrent Neural Networks: an Empirical Evaluation](https://mlsanity.com/paper/2103.07492/) (2021)
- [CLeaR: An Adaptive Continual Learning Framework for Regression Tasks](https://mlsanity.com/paper/2101.00926/) (2021)
- [Prototype-Guided Memory Replay for Continual Learning](https://mlsanity.com/paper/2108.12641/) (2021)
- [Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How](https://mlsanity.com/paper/2311.04898/) (2023)
- [Order parameters and phase transitions of continual learning in deep neural networks](https://mlsanity.com/paper/2407.10315/) (2024)
- [Bayesian Continual Learning via Spiking Neural Networks](https://mlsanity.com/paper/2208.13723/) (2022)
- [When Meta-Learning Meets Online and Continual Learning: A Survey](https://mlsanity.com/paper/2311.05241/) (2023)
- [Statistical mechanics of continual learning: variational principle and mean-field potential](https://mlsanity.com/paper/2212.02846/) (2022)
- [Bayesian continual learning and forgetting in neural networks](https://mlsanity.com/paper/2504.13569/) (2025)
- [Learning to Continually Learn with the Bayesian Principle](https://mlsanity.com/paper/2405.18758/) (2024)
- [Variational Continual Test-Time Adaptation](https://mlsanity.com/paper/2402.08182/) (2024)
- [On out-of-distribution detection with Bayesian neural networks](https://mlsanity.com/paper/2110.06020/) (2021)
- [Weight Factorization and Centralization for Continual Learning in Speech Recognition](https://mlsanity.com/paper/2506.16574/) (2025)
- [IBCL: Zero-shot Model Generation under Stability-Plasticity Trade-offs](https://mlsanity.com/paper/2305.14782/) (2023)
- [Approximate Bayesian Class-Conditional Models under Continuous Representation Shift](https://mlsanity.com/paper/2305.19076/) (2023)
- [Continual learning with the neural tangent ensemble](https://mlsanity.com/paper/2408.17394/) (2024)
- [Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings](https://mlsanity.com/paper/2410.20768/) (2024)
- [Autoencoder-Based Hybrid Replay for Class-Incremental Learning](https://mlsanity.com/paper/2505.05926/) (2025)
- [Streaming LifeLong Learning With Any-Time Inference](https://mlsanity.com/paper/2301.11892/) (2023)
- [Robust Importance Sampling for Error Estimation in the Context of Optimal Bayesian Transfer Learning](https://mlsanity.com/paper/2109.02150/) (2021)
- [Attack On Prompt: Backdoor Attack in Prompt-Based Continual Learning](https://mlsanity.com/paper/2406.19753/) (2024)
- [Continual Learning With Quasi-Newton Methods](https://mlsanity.com/paper/2503.19939/) (2025)
- [The Bayesian Approach to Continual Learning: An Overview](https://mlsanity.com/paper/2507.08922/) (2025)

## Abstract (arXiv)

Some machine learning applications require continual learning - where data comes in a sequence of datasets, each is used for training and then permanently discarded. From a Bayesian perspective, continual learning seems straightforward: Given the model posterior one would simply use this as the prior for the next task. However, exact posterior evaluation is intractable with many models, especially with Bayesian neural networks (BNNs). Instead, posterior approximations are often sought. Unfortunately, when posterior approximations are used, prior-focused approaches do not succeed in evaluations designed to capture properties of realistic continual learning use cases. As an alternative to prior-focused methods, we introduce a new approximate Bayesian derivation of the continual learning loss. Our loss does not rely on the posterior from earlier tasks, and instead adapts the model itself by changing the likelihood term. We call these approaches likelihood-focused. We then combine prior- and likelihood-focused methods into one objective, tying the two views together under a single unifying framework of approximate Bayesian continual learning.