# Memory Replay GANs: learning to generate images from new categories without forgetting

arXiv: https://arxiv.org/abs/1809.02058 · HTML: https://mlsanity.com/paper/1809.02058/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 30 of 77)

- [Rainbow Memory: Continual Learning with a Memory of Diverse Samples](https://mlsanity.com/paper/2103.17230/) (2021)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [On Learning the Geodesic Path for Incremental Learning](https://mlsanity.com/paper/2104.08572/) (2021)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning](https://mlsanity.com/paper/2101.00407/) (2021)
- [Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference](https://mlsanity.com/paper/2110.10031/) (2021)
- [Continual Learning via Local Module Composition](https://mlsanity.com/paper/2111.07736/) (2021)
- [R-DFCIL: Relation-Guided Representation Learning for Data-Free Class Incremental Learning](https://mlsanity.com/paper/2203.13104/) (2022)
- [Lifelong Person Re-Identification via Adaptive Knowledge Accumulation](https://mlsanity.com/paper/2103.12462/) (2021)
- [ACIL: Analytic Class-Incremental Learning with Absolute Memorization and Privacy Protection](https://mlsanity.com/paper/2205.14922/) (2022)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning](https://mlsanity.com/paper/2403.18886/) (2024)
- [Continually Learning Self-Supervised Representations with Projected Functional Regularization](https://mlsanity.com/paper/2112.15022/) (2021)
- [Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation](https://mlsanity.com/paper/2203.14098/) (2022)
- [Learnability and Algorithm for Continual Learning](https://mlsanity.com/paper/2306.12646/) (2023)
- [GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning](https://mlsanity.com/paper/2102.07868/) (2021)
- [Multi-Domain Incremental Learning for Semantic Segmentation](https://mlsanity.com/paper/2110.12205/) (2021)
- [Continual Learning Based on OOD Detection and Task Masking](https://mlsanity.com/paper/2203.09450/) (2022)
- [On the importance of cross-task features for class-incremental learning](https://mlsanity.com/paper/2106.11930/) (2021)
- [Lifelong Teacher-Student Network Learning](https://mlsanity.com/paper/2107.04689/) (2021)
- [Exploring Continual Learning of Diffusion Models](https://mlsanity.com/paper/2303.15342/) (2023)
- [Piggyback GAN: Efficient Lifelong Learning for Image Conditioned Generation](https://mlsanity.com/paper/2104.11939/) (2021)
- [Do Not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting](https://mlsanity.com/paper/2102.01906/) (2021)
- [Continual Evidential Deep Learning for Out-of-Distribution Detection](https://mlsanity.com/paper/2309.02995/) (2023)
- [Continual learning in cross-modal retrieval](https://mlsanity.com/paper/2104.06806/) (2021)
- [Cost-effective On-device Continual Learning over Memory Hierarchy with Miro](https://mlsanity.com/paper/2308.06053/) (2023)
- [A Memory Transformer Network for Incremental Learning](https://mlsanity.com/paper/2210.04485/) (2022)
- [Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection](https://mlsanity.com/paper/2209.09839/) (2022)

## Abstract (arXiv)

Previous works on sequential learning address the problem of forgetting in discriminative models. In this paper we consider the case of generative models. In particular, we investigate generative adversarial networks (GANs) in the task of learning new categories in a sequential fashion. We first show that sequential fine tuning renders the network unable to properly generate images from previous categories (i.e. forgetting). Addressing this problem, we propose Memory Replay GANs (MeRGANs), a conditional GAN framework that integrates a memory replay generator. We study two methods to prevent forgetting by leveraging these replays, namely joint training with replay and replay alignment. Qualitative and quantitative experimental results in MNIST, SVHN and LSUN datasets show that our memory replay approach can generate competitive images while significantly mitigating the forgetting of previous categories.