# Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning

arXiv: https://arxiv.org/abs/2202.10203 · HTML: https://mlsanity.com/paper/2202.10203/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 21 Feb 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Gradient based sample selection for online continual learning](https://mlsanity.com/paper/1903.08671/) (2019)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [On Tiny Episodic Memories in Continual Learning](https://mlsanity.com/paper/1902.10486/) (2019)
- [Semantic Drift Compensation for Class-Incremental Learning](https://mlsanity.com/paper/2004.00440/) (2020)
- [Reinforced Continual Learning](https://mlsanity.com/paper/1805.12369/) (2018)
- [Using Hindsight to Anchor Past Knowledge in Continual Learning](https://mlsanity.com/paper/2002.08165/) (2020)
- [Conditional Channel Gated Networks for Task-Aware Continual Learning](https://mlsanity.com/paper/2004.00070/) (2020)
- [Continual Learning via Neural Pruning](https://mlsanity.com/paper/1903.04476/) (2019)
- [Rethinking Experience Replay: a Bag of Tricks for Continual Learning](https://mlsanity.com/paper/2010.05595/) (2020)
- [iTAML: An Incremental Task-Agnostic Meta-learning Approach](https://mlsanity.com/paper/2003.11652/) (2020)
- [Task Agnostic Continual Learning Using Online Variational Bayes](https://mlsanity.com/paper/1803.10123/) (2018)
- [Selfless Sequential Learning](https://mlsanity.com/paper/1806.05421/) (2018)
- [SPACE: Structured Compression and Sharing of Representational Space for Continual Learning](https://mlsanity.com/paper/2001.08650/) (2020)
- [In Defense of the Learning Without Forgetting for Task Incremental Learning](https://mlsanity.com/paper/2107.12304/) (2021)
- [Variational Bayesian Dropout with a Hierarchical Prior](https://mlsanity.com/paper/1811.07533/) (2018)

## Built on by (most cited first, 29 of 29)

- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [RanPAC: Random Projections and Pre-trained Models for Continual Learning](https://mlsanity.com/paper/2307.02251/) (2023)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning](https://mlsanity.com/paper/2403.18886/) (2024)
- [CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Models](https://mlsanity.com/paper/2403.19137/) (2024)
- [Sparsity and Heterogeneous Dropout for Continual Learning in the Null Space of Neural Activations](https://mlsanity.com/paper/2203.06514/) (2022)
- [NPCL: Neural Processes for Uncertainty-Aware Continual Learning](https://mlsanity.com/paper/2310.19272/) (2023)
- [Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure](https://mlsanity.com/paper/2403.07292/) (2024)
- [KOPPA: Improving Prompt-based Continual Learning with Key-Query Orthogonal Projection and Prototype-based One-Versus-All](https://mlsanity.com/paper/2311.15414/) (2023)
- [COPAL: Continual Pruning in Large Language Generative Models](https://mlsanity.com/paper/2405.02347/) (2024)
- [Continual Learning on CLIP via Incremental Prompt Tuning with Intrinsic Textual Anchors](https://mlsanity.com/paper/2505.20680/) (2025)
- [The Importance of Being Lazy: Scaling Limits of Continual Learning](https://mlsanity.com/paper/2506.16884/) (2025)
- [Exclusive Supermask Subnetwork Training for Continual Learning](https://mlsanity.com/paper/2210.10209/) (2022)
- [Generalized Few-Shot Continual Learning with Contrastive Mixture of Adapters](https://mlsanity.com/paper/2302.05936/) (2023)
- [Premonition: Using Generative Models to Preempt Future Data Changes in Continual Learning](https://mlsanity.com/paper/2403.07356/) (2024)
- [Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Experts](https://mlsanity.com/paper/2506.21035/) (2025)
- [Continual Learning via Manifold Expansion Replay](https://mlsanity.com/paper/2310.08038/) (2023)
- [Model Inversion with Layer-Specific Modeling and Alignment for Data-Free Continual Learning](https://mlsanity.com/paper/2510.26311/) (2025)
- [Never Explore Repeatedly in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2308.09909/) (2023)
- [Variational Bayes for Federated Continual Learning](https://mlsanity.com/paper/2405.14291/) (2024)
- [Knowledge Transfer-Driven Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2306.10942/) (2023)
- [Hierarchically Gated Experts for Efficient Online Continual Learning](https://mlsanity.com/paper/2412.17188/) (2024)
- [Learning Dynamic Representations via An Optimally-Weighted Maximum Mean Discrepancy Optimization Framework for Continual Learning](https://mlsanity.com/paper/2501.12121/) (2025)
- [Continual learning via probabilistic exchangeable sequence modelling](https://mlsanity.com/paper/2503.20725/) (2025)
- [Structural Abstraction as an Inductive Bias for Non-Stationary Language Model Training](https://mlsanity.com/paper/2603.17198/) (2026)
- [Enhancing Continual Learning of Vision-Language Models via Dynamic Prefix Weighting](https://mlsanity.com/paper/2604.18075/) (2026)
- [Geometry of Forgetting: Representation Flux in Continual Learning](https://mlsanity.com/paper/2608.15854/) (2026)
- [Probabilistic Multi-Source Data Selection via Learnable Categorical Priors: Variance Reduction and Guaranteed Derandomization](https://mlsanity.com/paper/2610.14942/) (2026)
- [SPARS: A Stability-Plasticity-Aware Region Selection Framework for Continual Learning](https://mlsanity.com/paper/2610.19876/) (2026)

## Abstract (arXiv)

Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered. Existing CL approaches often keep a buffer of previously-seen samples, perform knowledge distillation, or use regularization techniques towards this goal. Despite their performance, they still suffer from interference across tasks which leads to catastrophic forgetting. To ameliorate this problem, we propose to only activate and select sparse neurons for learning current and past tasks at any stage. More parameters space and model capacity can thus be reserved for the future tasks. This minimizes the interference between parameters for different tasks. To do so, we propose a Sparse neural Network for Continual Learning (SNCL), which employs variational Bayesian sparsity priors on the activations of the neurons in all layers. Full Experience Replay (FER) provides effective supervision in learning the sparse activations of the neurons in different layers. A loss-aware reservoir-sampling strategy is developed to maintain the memory buffer. The proposed method is agnostic as to the network structures and the task boundaries. Experiments on different datasets show that our approach achieves state-of-the-art performance for mitigating forgetting.