# Pseudo-Rehearsal for Continual Learning with Normalizing Flows

arXiv: https://arxiv.org/abs/2007.02443 · HTML: https://mlsanity.com/paper/2007.02443/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2020.

## Built on by (most cited first, 10 of 10)

- [Task-agnostic Continual Learning with Hybrid Probabilistic Models](https://mlsanity.com/paper/2106.12772/) (2021)
- [Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal](https://mlsanity.com/paper/2110.09695/) (2021)
- [Continual Learning Approaches for Anomaly Detection](https://mlsanity.com/paper/2212.11192/) (2022)
- [Revisiting Catastrophic Forgetting in Class Incremental Learning](https://mlsanity.com/paper/2107.12308/) (2021)
- [GUIDE: Guidance-based Incremental Learning with Diffusion Models](https://mlsanity.com/paper/2403.03938/) (2024)
- [Looking through the past: better knowledge retention for generative replay in continual learning](https://mlsanity.com/paper/2309.10012/) (2023)
- [Elastic Multi-Gradient Descent for Parallel Continual Learning](https://mlsanity.com/paper/2401.01054/) (2024)
- [Adapt & Align: Continual Learning with Generative Models Latent Space Alignment](https://mlsanity.com/paper/2312.13699/) (2023)
- [Continual learning via probabilistic exchangeable sequence modelling](https://mlsanity.com/paper/2503.20725/) (2025)
- [Deep Generative Continual Learning using Functional LoRA: FunLoRA](https://mlsanity.com/paper/2510.02631/) (2025)

## Abstract (arXiv)

Catastrophic forgetting (CF) happens whenever a neural network overwrites past knowledge while being trained on new tasks. Common techniques to handle CF include regularization of the weights (using, e.g., their importance on past tasks), and rehearsal strategies, where the network is constantly re-trained on past data. Generative models have also been applied for the latter, in order to have endless sources of data. In this paper, we propose a novel method that combines the strengths of regularization and generative-based rehearsal approaches. Our generative model consists of a normalizing flow (NF), a probabilistic and invertible neural network, trained on the internal embeddings of the network. By keeping a single NF conditioned on the task, we show that our memory overhead remains constant. In addition, exploiting the invertibility of the NF, we propose a simple approach to regularize the network's embeddings with respect to past tasks. We show that our method performs favorably with respect to state-of-the-art approaches in the literature, with bounded computational power and memory overheads.