# Overcoming Catastrophic Forgetting by Generative Regularization

arXiv: https://arxiv.org/abs/1912.01238 · HTML: https://mlsanity.com/paper/1912.01238/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2019.

## Built on by (most cited first, 13 of 13)

- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Machine Unlearning: A Survey](https://mlsanity.com/paper/2306.03558/) (2023)
- [A General Framework for quantifying Aleatoric and Epistemic uncertainty in Graph Neural Networks](https://mlsanity.com/paper/2205.09968/) (2022)
- [Class-wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory Generation](https://mlsanity.com/paper/2307.03363/) (2023)
- [Overcoming Catastrophic Forgetting in Tabular Data Classification: A Pseudorehearsal-based approach](https://mlsanity.com/paper/2407.09039/) (2024)
- [CLVOS23: A Long Video Object Segmentation Dataset for Continual Learning](https://mlsanity.com/paper/2304.04259/) (2023)
- [Always Strengthen Your Strengths: A Drift-Aware Incremental Learning Framework for CTR Prediction](https://mlsanity.com/paper/2304.09062/) (2023)
- [Continual-learning for Modelling Low-Resource Languages from Large Language Models](https://mlsanity.com/paper/2601.05874/) (2026)
- [TegFormer: Topic-to-Essay Generation with Good Topic Coverage and High Text Coherence](https://mlsanity.com/paper/2212.13456/) (2022)
- [Memory-Efficient Continual Learning Object Segmentation for Long Video](https://mlsanity.com/paper/2309.15274/) (2023)
- [Machine Unlearning using a Multi-GAN based Model](https://mlsanity.com/paper/2407.18467/) (2024)
- [Enhancing knowledge retention for continual learning with domain-specific adapters and features gating](https://mlsanity.com/paper/2504.08613/) (2025)
- [Task Switching Without Forgetting via Proximal Decoupling](https://mlsanity.com/paper/2604.18857/) (2026)

## Abstract (arXiv)

In this paper, we propose a new method to overcome catastrophic forgetting by adding generative regularization to Bayesian inference framework. Bayesian method provides a general framework for continual learning. We could further construct a generative regularization term for all given classification models by leveraging energy-based models and Langevin-dynamic sampling to enrich the features learned in each task. By combining discriminative and generative loss together, we empirically show that the proposed method outperforms state-of-the-art methods on a variety of tasks, avoiding catastrophic forgetting in continual learning. In particular, the proposed method outperforms baseline methods over 15% on the Fashion-MNIST dataset and 10% on the CUB dataset