# Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set Recognition

arXiv: https://arxiv.org/abs/1905.12019 · HTML: https://mlsanity.com/paper/1905.12019/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 26 of 26)

- [Continual Learning via Bit-Level Information Preserving](https://mlsanity.com/paper/2105.04444/) (2021)
- [CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparability](https://mlsanity.com/paper/2110.03331/) (2021)
- [Incremental Generalized Category Discovery](https://mlsanity.com/paper/2304.14310/) (2023)
- [ICICLE: Interpretable Class Incremental Continual Learning](https://mlsanity.com/paper/2303.07811/) (2023)
- [Task-agnostic Continual Learning with Hybrid Probabilistic Models](https://mlsanity.com/paper/2106.12772/) (2021)
- [Out-Of-Distribution Detection In Unsupervised Continual Learning](https://mlsanity.com/paper/2204.05462/) (2022)
- [Class-Incremental Learning with Repetition](https://mlsanity.com/paper/2301.11396/) (2023)
- [Raising the Bar on the Evaluation of Out-of-Distribution Detection](https://mlsanity.com/paper/2209.11960/) (2022)
- [Probabilistic Circuits That Know What They Don't Know](https://mlsanity.com/paper/2302.06544/) (2023)
- [Joint Diffusion models in Continual Learning](https://mlsanity.com/paper/2411.08224/) (2024)
- [A Comprehensive Review of Trends, Applications and Challenges In Out-of-Distribution Detection](https://mlsanity.com/paper/2209.12935/) (2022)
- [Neural Architecture Search of Deep Priors: Towards Continual Learning without Catastrophic Interference](https://mlsanity.com/paper/2104.06788/) (2021)
- [A Procedural World Generation Framework for Systematic Evaluation of Continual Learning](https://mlsanity.com/paper/2106.02585/) (2021)
- [Generative Negative Replay for Continual Learning](https://mlsanity.com/paper/2204.05842/) (2022)
- [Looking through the past: better knowledge retention for generative replay in continual learning](https://mlsanity.com/paper/2309.10012/) (2023)
- [Managing the unknown: a survey on Open Set Recognition and tangential areas](https://mlsanity.com/paper/2312.08785/) (2023)
- [Out-of-distribution forgetting: vulnerability of continual learning to intra-class distribution shift](https://mlsanity.com/paper/2306.00427/) (2023)
- [Class-Prototype Conditional Diffusion Model with Gradient Projection for Continual Learning](https://mlsanity.com/paper/2312.06710/) (2023)
- [Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer](https://mlsanity.com/paper/2502.20124/) (2025)
- [Flashbacks to Harmonize Stability and Plasticity in Continual Learning](https://mlsanity.com/paper/2506.00477/) (2025)
- [Masked Autoencoders are Efficient Continual Federated Learners](https://mlsanity.com/paper/2306.03542/) (2023)
- [Class incremental learning with probability dampening and cascaded gated classifier](https://mlsanity.com/paper/2402.01262/) (2024)
- [Resilience to the Flowing Unknown: an Open Set Recognition Framework for Data Streams](https://mlsanity.com/paper/2411.00876/) (2024)
- [Continual Causal Abstractions](https://mlsanity.com/paper/2212.12575/) (2022)
- [Slowing Down Forgetting in Continual Learning](https://mlsanity.com/paper/2411.06916/) (2024)
- [Deep evolving semi-supervised anomaly detection](https://mlsanity.com/paper/2412.00860/) (2024)

## Abstract (arXiv)

Modern deep neural networks are well known to be brittle in the face of unknown data instances and recognition of the latter remains a challenge. Although it is inevitable for continual-learning systems to encounter such unseen concepts, the corresponding literature appears to nonetheless focus primarily on alleviating catastrophic interference with learned representations. In this work, we introduce a probabilistic approach that connects these perspectives based on variational inference in a single deep autoencoder model. Specifically, we propose to bound the approximate posterior by fitting regions of high density on the basis of correctly classified data points. These bounds are shown to serve a dual purpose: unseen unknown out-of-distribution data can be distinguished from already trained known tasks towards robust application. Simultaneously, to retain already acquired knowledge, a generative replay process can be narrowed to strictly in-distribution samples, in order to significantly alleviate catastrophic interference.