# Continual Learning Using Bayesian Neural Networks

arXiv: https://arxiv.org/abs/1910.04112 · HTML: https://mlsanity.com/paper/1910.04112/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2019.

## Built on by (most cited first, 19 of 19)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Posterior Meta-Replay for Continual Learning](https://mlsanity.com/paper/2103.01133/) (2021)
- [Continual Learning for Recurrent Neural Networks: an Empirical Evaluation](https://mlsanity.com/paper/2103.07492/) (2021)
- [Sequoia: A Software Framework to Unify Continual Learning Research](https://mlsanity.com/paper/2108.01005/) (2021)
- [Prototype-Guided Memory Replay for Continual Learning](https://mlsanity.com/paper/2108.12641/) (2021)
- [Unsupervised Continual Learning in Streaming Environments](https://mlsanity.com/paper/2109.09282/) (2021)
- [Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning](https://mlsanity.com/paper/2308.02346/) (2023)
- [On the Robustness of Bayesian Neural Networks to Adversarial Attacks](https://mlsanity.com/paper/2207.06154/) (2022)
- [Tiny Machine Learning for Concept Drift](https://mlsanity.com/paper/2107.14759/) (2021)
- [Uncertainty Prioritized Experience Replay](https://mlsanity.com/paper/2506.09270/) (2025)
- [IBCL: Zero-shot Model Generation under Stability-Plasticity Trade-offs](https://mlsanity.com/paper/2305.14782/) (2023)
- [AdaptCL: Adaptive Continual Learning for Tackling Heterogeneity in Sequential Datasets](https://mlsanity.com/paper/2207.11005/) (2022)
- [Branch-Tuning: Balancing Stability and Plasticity for Continual Self-Supervised Learning](https://mlsanity.com/paper/2403.18266/) (2024)
- [Characterizing Continual Learning Scenarios and Strategies for Audio Analysis](https://mlsanity.com/paper/2407.00465/) (2024)
- [Continual Learning of Generative Models with Limited Data: From Wasserstein-1 Barycenter to Adaptive Coalescence](https://mlsanity.com/paper/2101.09225/) (2021)
- [Overcoming Domain Drift in Online Continual Learning](https://mlsanity.com/paper/2405.09133/) (2024)
- [A Self-Commissioning Edge Computing Method for Data-Driven Anomaly Detection in Power Electronic Systems](https://mlsanity.com/paper/2312.02661/) (2023)
- [The Bayesian Approach to Continual Learning: An Overview](https://mlsanity.com/paper/2507.08922/) (2025)
- [Forecasting Thermoacoustic Instabilities in Liquid Propellant Rocket Engines Using Multimodal Bayesian Deep Learning](https://mlsanity.com/paper/2107.06396/) (2021)

## Abstract (arXiv)

Continual learning models allow to learn and adapt to new changes and tasks over time. However, in continual and sequential learning scenarios in which the models are trained using different data with various distributions, neural networks tend to forget the previously learned knowledge. This phenomenon is often referred to as catastrophic forgetting. The catastrophic forgetting is an inevitable problem in continual learning models for dynamic environments. To address this issue, we propose a method, called Continual Bayesian Learning Networks (CBLN), which enables the networks to allocate additional resources to adapt to new tasks without forgetting the previously learned tasks. Using a Bayesian Neural Network, CBLN maintains a mixture of Gaussian posterior distributions that are associated with different tasks. The proposed method tries to optimise the number of resources that are needed to learn each task and avoids an exponential increase in the number of resources that are involved in learning multiple tasks. The proposed method does not need to access the past training data and can choose suitable weights to classify the data points during the test time automatically based on an uncertainty criterion. We have evaluated our method on the MNIST and UCR time-series datasets. The evaluation results show that our method can address the catastrophic forgetting problem at a promising rate compared to the state-of-the-art models.