# Improving and Understanding Variational Continual Learning

arXiv: https://arxiv.org/abs/1905.02099 · HTML: https://mlsanity.com/paper/1905.02099/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2019.

## Built on by (most cited first, 12 of 12)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Posterior Meta-Replay for Continual Learning](https://mlsanity.com/paper/2103.01133/) (2021)
- [Continual Learning via Sequential Function-Space Variational Inference](https://mlsanity.com/paper/2312.17210/) (2023)
- [Partitioned Variational Inference: A Framework for Probabilistic Federated Learning](https://mlsanity.com/paper/2202.12275/) (2022)
- [EVCL: Elastic Variational Continual Learning with Weight Consolidation](https://mlsanity.com/paper/2406.15972/) (2024)
- [Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process](https://mlsanity.com/paper/2108.12278/) (2021)
- [TyXe: Pyro-based Bayesian neural nets for Pytorch](https://mlsanity.com/paper/2110.00276/) (2021)
- [On Sequential Bayesian Inference for Continual Learning](https://mlsanity.com/paper/2301.01828/) (2023)
- [IF2Net: Innately Forgetting-Free Networks for Continual Learning](https://mlsanity.com/paper/2306.10480/) (2023)
- [Overcome Anterograde Forgetting with Cycled Memory Networks](https://mlsanity.com/paper/2112.02342/) (2021)
- [Continual Invariant Risk Minimization](https://mlsanity.com/paper/2310.13977/) (2023)
- [Sample Weight Estimation Using Meta-Updates for Online Continual Learning](https://mlsanity.com/paper/2401.15973/) (2024)

## Abstract (arXiv)

In the continual learning setting, tasks are encountered sequentially. The goal is to learn whilst i) avoiding catastrophic forgetting, ii) efficiently using model capacity, and iii) employing forward and backward transfer learning. In this paper, we explore how the Variational Continual Learning (VCL) framework achieves these desiderata on two benchmarks in continual learning: split MNIST and permuted MNIST. We first report significantly improved results on what was already a competitive approach. The improvements are achieved by establishing a new best practice approach to mean-field variational Bayesian neural networks. We then look at the solutions in detail. This allows us to obtain an understanding of why VCL performs as it does, and we compare the solution to what an `ideal' continual learning solution might be.